Line a 900g loaf tin well with cling film, then scatter over the almonds (if you oil the tin before lining with film, it’ll help it keep in place smoothly).
Divide the condensed milk and double cream evenly between 2 bowls. Add the sherry to one, and the vanilla paste and food colouring, if using, to the other. Pop the vanilla bowl in the fridge for later, then whisk the sherry mixture until thick.
Spoon the sherry cream on top of the almonds and smooth the surface to level. Cover the tin with cling film, then freeze for about 1 hr until firm.
Whisk the vanilla cream until thick, then spoon over the frozen sherry cream layer. Make this layer as smooth as possible too, then put back into the freezer for another hr until firm.
To make the raspberry layer, soak the gelatine in a little cold water while you whizz half the raspberries in a food processor with the icing sugar and 2 tbsp water. Sieve into a small pan and heat gently to warm through. Once it starts to boil, remove from the heat, squeeze out the gelatine and add to the purée, stirring all the time, until the gelatine melts.
Crush half the remaining raspberries with a fork, then stir these into the purée with remaining whole raspberries and allow to cool. Spoon the raspberry mix evenly on top of the vanilla layer, press the sponge fingers lightly into it, then freeze until solid – overnight is best.
Remove the trifle from the freezer about 15 mins before serving. Tip the loaf tin upside down on a serving plate, then peel off the cling film and cut into neat slices using a sharp knife.