In a 4-quart saucepan or pot, bring the first 6 ingredients to a simmer. Stir slowly to dissolve the sugar in the liquid. Add the remaining ingredients except the lemon juice and the cheese. When the liquid comes to a boil, adjust the heat and simmer for about 20 minutes to marry the flavors and soften the fruit. Turn off the heat, then stir in the lemon juice. With a slotted spoon, remove the fruit to a bowl.
Bring the remaining syrup to a simmer and reduce until lightly thickened or when the bubbles formed on the top become small. Remove the syrup for the heat and when it is cool, pour it over the fruit. The compote can be held in the refrigerator for up to 4 days.
Serve the compote at room temperature with the cheese.