To remove or not to remove

Given that thickness in the space is limited, the only variable is the thermal conductivity difference between the XF and whatever you have in mind as a replacement.

I suspect the XF was used due to awkward spaces but if not then well fitted wood-fibre insulation will be a better option but won't be cheap

A little late to this particular party, but I'll assume you haven't had a fit of mania and done all the work in a week. :-)

Sounds like you're talking about a van conversion (as opposed to the renovation of a factory-built campervan), in which case the point you most need to consider is how, when and where you'll use the van, and whether you might benefit from having some cold(er) zones during hotter weather.

With the benefit of 20 years practical experience in temperatures ranging from -15°C to +40°C, I can tell you that it's a heck of a lot easier to add supplementary insulation for a winter trip than to try and remove it during in the summer, and it's primarily in the summer that you'll wish you had a spot in the vehicle that's 10°C cooler than the rest.

This spot will almost certainly be at floor level, so unless you need to block up some holes, I'd leave well enough alone, and invest instead in a decent piece of carpet for the exposed/living space. That'll give you comfort underfoot, as well as thermal and acoustic insulation, and you can configure/fit out the hidden spaces individually according to their future use.