Doctor Josh Dubnau explains that genes are responsible for memory in that they contain the raw instructions for memory. Experience determines how these instructions are assembled.
Genes are responsible for memories in the way that a recipe is responsible for a great soufflé. The genes contain the instructions, first for how to build a brain during development, and then the genes contain the instructions that that brain needs to function in an environment that allows learning and memory to happen. But, just the way a great chef takes a recipe and uses all of their prior experience to influence how light and fluffy the soufflé is, brains do the same thing. The genes are really just the raw instructions that impart upon brains the ability to perform in a way that we call learning or memory.