I don't really know if the idea of posh people making backroom deals is all that interesting - don't we pretty much take that for granted?
I am familiar with the sinister conspiracy theories, but short of going all Robert Anton Wilson or Umberto Eco about it and stacking every single one of the most outrageous claims on top of eachother and claiming them true (hell, throw some of Davic Icke's reptilians in there too), it doesn't strike me as a very interesting choice. A totally-out-there Zak McKracken style game based on real-world conspiracies could be a lot of fun though.
Even for the basic Dan Brown approach of name-dropping real world people and organizations into some tangled thriller web, Bilderberg seems quite dull - no matter its scope, it's mainly politics, money and grumpy people behind oversize desks.
Personally I'd go for the Bohemian Grove folks instead - though I'm sure there's a huge overlap with Bilderberg. Or maybe Skull & Bones. Not that I think they're anywhere near as sinister as Alex Jones likes to think, but at least those groups instantly evoke settings, visuals and esoteric historical connections that I'd love to explore in an adventure game form. But please do prove me wrong
Edit: That being said, after watching one of Glenn Beck's paranoid rants the other day, I actually went on Wikipedia to research the Fabian Society for possible game plot material
