"Any sufficiently advanced technology is indistinguishable from magic."
In the grim darkness of the 41st millennium, everyone thinks machines have souls.
Even worse, they very well might be right. Because that is just how weird everything is in the big blackness of the grimdark future.
This isn't really that absurd a belief, considering how we tend to anthropomorphise things like computers and cars - ascribing them personalities, habits and so on. The idea of Machine Spirits is what develops when you let this rampage unchecked by common sense among a technocratic & theocratic ruling class. However, what's important to know is that prior to humanity's downfall and the Dark Age of Technology, artificial intelligence was widespread in what was a typically utopian future. Given that the Adeptus Mechanicus have a hard-on for hoarding ancient technology, it's not unreasonable to assume that particularly old and/or complex machines do in fact think for themselves. This turns into nightmare fuel when you realize that Mars is honeycombed with factories, warehouses and other facilities in varying states of disuse - facilities that were first the battlegrounds between the different groups of survivors when the Dark Age of Technology came knocking, and then again between the loyal and heretical halves of the Mechanicus during the Horus Heresy. Hideous self-aware weapons of death still stalk the lower levels, possibly tainted by Chaos and definitely insane from 10,000 years of solitude and the machine equivalent of PTSD. (Look to what happened to Arkhan Land on his expeditions for instance.)
The Imperium at large knows little of this, but Machine Spirits are the reason put forward by the Mechanicus to have everyone maintain machines and treat them with respect. They generally vulgarize the complex subject for the plebs as a simple quid pro quo: take care of your gear, and it will take care of you in turn. And the best part? It works. Even if the average citizen has no idea what they're doing exactly, having their machines work correctly is motivation enough for everything a Techpriest tells them to do; from the simple daily ritual of cleaning to appease a Lasgun to the incredibly complex rites performed to reawaken the demanding and temperamental machine spirits of Titans.
There is also one other problem that one tends to forget--there actually are things called Machine Spirits. After the rebellion of the Men of Iron during the Age of Strife, Artificial Intelligence was declared "Abominable Intelligence" by decree of the Emperor of Mankind. To get around this, human brains were recycled and placed into the more complex machines (namely shipboard AIs and other cogitators, and presumably any robots who couldn't be replaced with servitors) which would then process the more complex information and control systems under the direction of a human. Being recycled from dead humans, they're understandably pretty cantankerous. Plus there are several signs that some advanced equipment, such as Land Raiders, Titans, and various old spaceships, have actual inorganic A.I. (albeit with an extremely specific scope of intelligence) These machines remain in operation despite -and in some cases, because of- the Mechanicus' inability to fully understand how they work. That said we have been given a pretty big glimpse: in The Unforgiven by Gav Thorpe, we learn that Space Marines too damaged for cybernetic reconstruction but aren't worthy of a Dreadnought are turned into the machine spirits for Land Raiders.
It is also implied sometimes (depending on the source) that Machine Spirits may be more real than first thought. There are examples of weapons that have no power source still being able to fire themselves. There was also at least one instance of a bunch of gears that were not connected to anything but each other (i.e. no motor) actually moving on their own after being talked to by a Tech-priest (maybe micro-motors? The Mechanicus is actually advanced). Or a gun that actually felt that it failed its master. Considering the Dark Age of Technology's feats, it is possible that dark age humans had successfully found a way to bend the warp into machines to give the machines their own "soul". This may also have helped cause the Iron War. After all, what does Chaos do with unprotected, unaware souls? Heh, yeah. Anyway, most "leftover" Machine Spirits tend to be a bit crazy, though, and using human brains merged with tech is safer because at least you know how the brain works. Hah, sort of.
In truth, and after checking most fluff material both from the Imperium's codices and Black Library novels you can find there is plenty of examples of Machine Spirits actively working along with Techpriests, Space Marines, Titan crews, etc, and they will show enough independence of thought to get mad if an enemy hurts their operators or know when to sacrifice themselves if there is no other way to get the task done. One such famous tale is the Land Raider known as Rynn's Might that belonged to the Crimson Fists Chapter, which survived the missile that leveled their fortress-monastery. Despite not having any crew on board, the machine spirit fought a solo war against an attacking Ork Warband, killing its Warboss and many of his followers overnight, before finally being destroyed... BUT IT SURVIVED! The machine spirit was recovered from the wrecked land raider by the chapter's techmarines and is currently awaiting a new vessel to house it
Some authors imply that the existing machine spirits are actually the remaining "Men of Gold", the only true and relatively benevolent machine spirits. These machine spirits are not particularly fond of helping their human caretakers as they have little reason to despite being extremely powerful and having full knowledge of the object they inhabit. Tech-priests rarely interact with these passive aggressive mechanical individuals and have almost no memory of the event, even so these spirits often require a lot of convincing just to make the on/off switch work.
Another interpretation is the machine spirit is the Dark Age of Technology's equivalent to Siri. It is an AI, but not a sentient one. STC technology was designed to be as inherently easy to use as possible, which would include making everything a smart device that could understand verbal commands. Not true AI, the smart devices could hear commands and try to orient themselves to do what their operator wanted, even if the right button wasn't being pushed. Now, keep in mind that the current Imperium is working on copies of copies of replicas of crudely reprinted versions of the original STC instructions (duplicated imperfectly by rote). There are still elements of the smart device listening and wanting to follow their human's commands, but they have a garbled understanding of vocabulary (or in another scenario, said human is speaking the incorrect language due to several thousand years of divergence). The Tech priests in turn have developed their rituals to be heard and understood by the machine spirits, unknowingly speaking keywords the device can try to decipher. In short, a Techpriest could be speaking instructions to Siri descendants in just about everything they do when it comes to tech.
There exists another possibility. Given how incredibly advanced humanity was, there could be micro-circuitry which, due to its sheer complexity, developed in such a way that it provided a sort of pseudo-brain for many pieces of technology in a sort of subconscious manner with a vague, general awareness. Techpriests might have discovered this on their own and found ways to interface and communicate with these semi-entities.
A final, stranger theory is that "machine spirits" are actually the work of the Void Dragon. He, and some Necrons, can directly control machinery. Combine this with how the creators of much of the more complicated machinery in the Imperium actually worship the Dragon under the guise of the Omnissiah and one starts to wonder if the Dragon has something to do with all of this. Perhaps a mutually beneficial and permanent deal with the Emperor to protect human tech from being corrupted by Data Daemons in exchange for passively feeding on it?
Hell it could be a case of all of the above depending on the individual piece of technology. Which would go a long way to explaining why the Mechanicus have such a hard time. The Rules keep changing on them drastically and they have no idea there's more than one set of rules.