I'm adding on to this to say that my x5 is the same. Without the Sega CD, games play fine, but with it attached, many games will freeze, or some will have graphical errors. Ram test is fine without the Sega CD, but Error 01 with it attached.
Additionally, when you try to use the x5 as a RAM cart with Sega CD games, it causes problems. For example, Popful Mail will display the Main CPU error l, which displays when RAM isn't formatted, but it displays it all garbled. Also, when viewing the Memory manager on the CD player menu, it seems to incorrectly report the space used on the Sega CD and the RAM cart itself, giving different values each time you go in.
This is all on Model 2 Genesis and Model 2 Sega CD, no 32X. I'm curious if I get a 32X if it will solve it. It basically seems like there's some kind of conflict when both units are attached.
Edit:
It looks like the battery for the onboard Sega CD RAM may be done. It holds a charge for a bit of time after the unit has been powered off, but will need a format due to a wipe, if left powered off for long enough.
I have a feeling replacing that battery may solve the issue, as another thread seemed to remark as such for a similar if not the same issue. I will report back with what I find.