@tokyojon - You must have missed some, im sure of it, although.. i deleted my old toon that was stuck on that part and re-made it, and i killed all the enemies in the castle, and the door actually opened o:
I think it may be a bug. This happened to my human rogue. I cleared as much of the castle as possible before going to the main hall. After the battle with sir gilmore he ran to the main double door entrance, but the side door leading the the other half of the castle remained locked.
Luckily i saved the game right when i woke up naked in my bedroom before. I reloaded, and did the exact same thing i did before (clearing the castle) and this time the side door was unlocked. I think it just bugged :P