Bed bugs, lint, or dirt?

It doesn't sound like bed bugs. They are the size of apple seeds and are reddish in color. You may be having an allergic reaction to the memory foam and the lint is coincidence. (My best guess is that you are seeing lint.)