People claim genes influence personality. The meaning of this isn't explained very clearly. Do Walmarts influence people to drive to one location instead of another? Sort of, but that doesn't mean Walmarts are limiting our free will or controlling us, and we can certainly choose not to drive to Walmart ever again.
But I'm going to focus on something else: Why do they think personality even exists at a hardware level or low level of software? I've never seen any genes-influence-personality advocates answer or even discuss this question.
Humans are complicated. They have a lot of mental stuff. A generic word for mental stuff is "ideas". Personality isn't generic, it's a category of mental stuff (category of ideas).
We create categories for our discussions and thinking which help us make sense of people. Instead of just saying "a person is a bunch of ideas" we come up with some organization and structure to help us make sense of it. We want chunks we can deal with, like personality, rather than a hugely complex chaos that we can't work with.
That's fine. Categorizing a personality idea differently than an idea about how to do arithmetic is a reasonably functional distinction. It offers us some way to mentally organize a person into parts and start dealing with them.
But we should keep in mind it's a category we made up to try to make sense of humans. It's a structure we imposed on people, and just because it's useful doesn't mean it's accurate. There are other possible ways to mentally categorize a human intelligence into different parts that don't rely on the concept of personality. Not all reasonable ways of categorizing complex stuff are the way the complex stuff is actually internally organized. They can't be, since there's a bunch of categorization options and only one actual internal structure.
How are human minds actually structured internally? The claims of some "scientists" notwithstanding, we don't really know a lot about that. Most of what we know is that it has to be a structure which is compatible with stuff humans do, such as use math and language, do science and chess, play football and soccer, enjoy art and music, write poems and prose. That rules out minds being a totally disorganized chaos. And it indicates humans can create new knowledge, which means evolution of ideas is taking place.
The approach people use is like looking at web browser software and assuming its written in object oriented programming with webpages, links, paragraphs, words, letters and buttons as objects. It could be. That's a possible way to organize a browser. But it doesn't have to be. The code for a browser could also be structured with a different hierarchy of objects, or with a different style of programming entirely that doesn't even use objects.
You can't easily tell how complex software is programmed by looking at what it does. You can mentally categorize a browser into parts like the menus, the URL bar, the status bar, and the webpage which has sub-parts like paragraphs, links, buttons, etc. That's fine as a way to think about it. But it doesn't mean that's how the software is organized internally. This applies to any sort of complex stuff with unseen internal structure, whether it's software or not. It's really hard to look at functionality and think you know internal structure because there's many structures which achieve the same or similar functional results.
For another example, suppose you have a machine which does multiplication (I previously discussed this example, and knowledge structure). Do you know what's going on internally? No. There's many different ways multiplication can be done, such as with a lookup table, a loop with repeated addition, recursion with repeated addition, or sending a text message to an employee in India and relaying his answer. It's great that you have a mental model of how to multiply. And your model will be useful for thinking about this machine. But that doesn't mean the machine's internal structure actually has anything to do with your mental model of one way that multiplication can be done.
So to recap, we don't know much of the details of how minds are structured internally. "Personality" is an organizational concept we find reasonably useful for thinking about minds. But that doesn't mean minds are actually organized that way – personality could just be an emergent property, an implication of some sort, or an approximate fudge which is similar to some other thing that actually exists. Or personality could be part of minds, but only at high levels of abstraction, not at the hardware level and the low-abstraction software level where genes could potentially influence or control things. It's unsafe to assume the actual structure of minds matches the mental categories we've created to help us deal with people.
People think it's uncontroversial and basically settled that genes influence personality. But we don't know that, and they might not, and personality might not even be part of the actual structure of human minds at all.