Complex home automation system can fail in many ways, but the most important thing that I really don't want it to be failing is security. Making everything connected just opens too many new possible attack vectors.
Of course the automation and "smarts" can also improve home security. Monitoring the activity and access can tell you immediately if there is something wrong. A smart system can also mimic normal living conditions automatically while you are away making it not too apparent that the house is empty.
Being involved in software industry long enough has just proven to me that every system can be hacked. Of course there are many things to make it harder, but it usually also means making your own life harder due to those limitations defeating the purpose.
Maintaining the security of the software also requires keeping the systems up to date. First, this update cycle can be inconvenient when you need to wait for the simplest thing due to the update being installed. If you keep it manual you need to pay attention to update notifications and remember to do the actual updates. Updates also tend to have a habit of breaking other things that require additional effort after each update to make sure everything still works as intended.