Combine all the ingredients except the apricots and raisins with ½ cup water in a medium saucepan and bring to a boil.
Cook over moderate heat, stirring occasionally, until just thickened, about 15 minutes. Stir in the apricots and raisins and cook until softened, about 2 minutes. Discard the rosemary.
Transfer the relish to a serving bowl. Cool to room temperature, then chill. Stir in more apricots and serve.
Make-Ahead: Make the relish, cover, and refrigerate for up to 4 days. Bring to room temperature before serving.