Got 3 pieces of set armor thus far, all from the various nutty undead fellows in Bedlam (can remember Lady Jennifyr particularly, but also off some others)... haven't seen any parts from Miasma, Lurg or Swoop myself but according to others at least Swoop does drop 'em. They seem to drop seriously rarely, like daemon bone bits or worse.

Sadly all my parts are for different sets...