| Home > My Projects > Home Automation > Home Assistant |
It wasn't long in my searching before I started coming across the various open source home automation systems. And open source was an absolute requirement as one cannot rely long term on any closed source software because support could end at any point. Even if open source support ends, the product and code remains allowing continuing use and future modifications. After evaluating the various options, I decided to base my system around Home Assistant.
First, About The Others |
There are a number of open source home automation platforms out there and I did check them out before deciding on Home Assistant. My thoughts on a few of them and why I didn't go that direction are below.
OpenHAB is extremely popular and was the first system I considered. I really liked the pseudo-scripting language used to write automations as that is always how I have considered automation to work. Exposing devices, sensors and inputs to a language you can use to control them. However it became a full stop when I found the system runs in Java. Java is a world of hurt due to differences in runtimes, which is a huge irony for a "write once run anywhere" platform. Indeed when I reviewed the installation documentation there were three different Java runtimes listed, with descriptions of support levels for each. Nope.
Next up was Domoticz. I can't put my finger on it, but Domoticz just struck me as weird. I liked the fact it is written in C and compiled to a native binary. Efficient, and low memory use. Yet the things that weirded me out were: at the time, poor documentation in an incomplete wiki (I note the wiki has be dramatically improved since I first looked), "manual" link at top of website home page just lead to a PDF dated Feb. 2015 and dead links on the home page. I know Domoticz is still in development with an active community however it just felt like development stopped years ago.
While not geared specifically towards home automation, Node-RED is an excellent automation/rules platform with an easy to use drag and drop interface. Except that it runs in node.js. Full stop. Sorry, but a JavaScript server side runtime that runs on Chrome's scripting engine? There's at least 2 things wrong with that sentence. 40 years from now, none of that will exist.
I also ran across OpenNetHome and ago control. Looks like development on both those projects stalled several years ago.
Home Assistant |
The Home Assistant documentation can be a confusing place to begin and there was definitely a steep learning curve. For example, for a long time the Python file responsible for an integration ("integrations" are a device class, a method of control, a sensor type, etc.) was shown in a link on the integration's page, which incorrectly lead me to believe that in order to set up Home Assistant one would have to manually include, download or otherwise manage these Python files. This is not the case as they are included with the Home Assistant distribution. I also found it very difficult to know exactly what to include in my YAML configuration files because there are multiple ways to express things in YAML and the documentation often switches between them. Without the consistency, in the beginning I found myself chasing YAML errors whenever I made a configuration change.
Installation Method |
Part of the confusion with Home Assistant is that there are so many installation methods, and the documentation tends to push you towards installing what was formally called "Hass.io" but is now called "Home Assistant. This is an Raspberry Pi image, running a limited OS, which runs Home Assistant Core (the name for the actual Home Assistant software) in Docker, with additional software to manage it all and add-on/plugin capability. There is Home Assistant Supervised which is Hass.io but running on your own OS choice (ie. Ubuntu). Then there is Home Assistant Core, which is the Home Assistant software running on your Linux, via either a direct install method, or in a Python venv (virtual environment). And of course, there is a Home Assistant (Core) Docker container. I'm sure there are other installation methods. These are the ones I found in 5 minutes of searching while writing this. Note that most of the HA installation documentation makes constant references to Raspberry Pi, even though the installation instructions are for generic Linux installation.
I made the decision to install Home Assistant as close to "bare metal" within Linux as possible. So I installed Home Assistant Core first directly, but then later moved to a Python Virtual Environment for unknown reasons. Honestly I think it just happened during a reinstall because I followed that set of instructions. In my journey I did end up reinstalling Home Assistant (Core) several times.
When I refer to Home Assistant, HA or HASS here with regards to my installation, I am really referring to Home Assistant Core running in a Python venv.
The Learning Curve |
I've been in IT professionally since 1997 integrating systems from a single user small business to thousands of users across multiple locations in multiple countries and I have to say that the learning curve for Home Assistant was one of the steepest I have ever encountered.
The most difficult process was wrapping my head around YAML and how it is used in Home Assistant. YAML (YAML Ain't Markup Language....cute). YAML relies on whitespace heavily, not tabs. And thus anyone used to programming in languages such as VB, C and similar will immediately write a lot of invalid YAML by accidentally using tab instead of space to line up the columns. Home Assistant will for the most part let you know where the invalid YAML is when it loads the files yet it is easy to get into a situation where your error is not visible by eye, and higher up in the file than Home Assistant reports. So it is best to use a YAML aware editor (begrudgingly I use VSCode which could be an entirely different rant in itself) that can make structure errors very obvious, with automatic "linting". The official YAML spec is not helpful.
How YAML is used within Home Assistant can be confusing as there frankly aren't a lot of good examples in the documentation. For example, if you Google "home assistant light" you end up, quite reasonably, at the light integration page which looks like it shows you how to define a light in Home Assistant's YAML config. Except it doesn't. Instead it documents how to use HA services to control a light, and provides a YAML automation to turn a light on when a motion sensor is triggered. No information how to actually add a light to HA without even a link to find such information. It is only through searching or manually looking through the documentation where one finds the YAML needed to add a light.
Now looking at the YAML for MQTT light (one must first understand what an "MQTT light" actually is...it is a light controlled by the MQTT protocol) it presents an YAML configuration example (I have chopped it heavily to use as an example):
# Example configuration.yaml entry
mqtt:
light:
- name: "Room Light 1"
state_topic: "room/light1/status"
command_topic: "room/light1/switch"
qos: 0
payload_on: "ON"
payload_off: "OFF"
|
I'd say that is fairly straightforward (when one understands MQTT) and I was able to add a light to my HA configuration.
Then I added a second light:
# Example configuration.yaml entry
mqtt:
light:
- name: "Room Light 1"
state_topic: "room/light1/status"
command_topic: "room/light1/switch"
qos: 0
payload_on: "ON"
payload_off: "OFF"
light:
- name: "Another Room Light"
state_topic: "room/light2/status"
command_topic: "room/light2/switch"
qos: 0
payload_on: "ON"
payload_off: "OFF"
|
And upon restarting Home Assistant, immediately received a "duplicate key: light" error, and Home Assistant refused to start.
Experienced YAML herders have already seen the error. One can't define multiple lights starting by "light:" as one would expect. Those keys tell HA that you are defining a light, with values to follow. Except each key must be unique. So you need to either represent multiples lights in one of the two ways below (there may be other options as well):
# Example configuration.yaml entry
mqtt:
light roomlight1:
- name: "Room Light 1"
state_topic: "room/light1/status"
command_topic: "room/light1/switch"
qos: 0
payload_on: "ON"
payload_off: "OFF"
light anotherroomlight:
- name: "Another Room Light"
state_topic: "room/light2/status"
command_topic: "room/light2/switch"
qos: 0
payload_on: "ON"
payload_off: "OFF"
|
# Example configuration.yaml entry
mqtt:
light:
- name: "Room Light 1"
state_topic: "room/light1/status"
command_topic: "room/light1/switch"
qos: 0
payload_on: "ON"
payload_off: "OFF"
- name: "Another Room Light"
state_topic: "room/light2/status"
command_topic: "room/light2/switch"
qos: 0
payload_on: "ON"
payload_off: "OFF"
|
The HA YAML configuration documentation does cover this somewhat and has been improved significantly since I first looked it up. However it still uses phrases such as "The following example shows nesting a collection of mappings in a mapping. " which now that I'm used to YAML makes perfect sense but at the time, read as near gibberish.
Primarily though, my initial difficulties in configuring Home Assistant where mainly due to just not having some moderate examples. YAML definitions were either shown as single items, or in complex setups spread across multiple YAML files in someone's Git repository. It would have been very helpful to see an example YAML configuration showing a few lights, switches and sensors with automation or two.
Since then though, the documentation has increased leaps and bounds in this area so I have no doubt that if I had started now, it would have been much easier.
Home Assistant has its own language which took some getting used to. "Platforms", "Integrations", "Devices", "States", "Attributes", "Entities". A "Platform" is a way to communicate with a specific piece of hardware through some sort of method. For example, the MQTT platform means that something communicates via the MQTT protocol. The "Hue" platform communicates with Philips Hue devices. Under Platforms are "Integrations" which are individual items such as a light, sensor, switch, etc. "Devices" are both another way of referring to Integrations and also not. Some things are only Devices, some Integrations are also Devices. The "Device" description was added primarily to help creating certain types of automations. An "Entity" is any sort of object in HA whether that be an automation, a sensor, an integration, a script. Entities have a "State" which shows the primary property of the entity. For example, a switch entity has states of "ON" or "OFF". "Attributes" are additional secondary properties of an entity. A light entity may have its RGB colour as an attribute.
"Templates" were another source of confusion because in my mind, a "template" is a blank document or framework that accepts values from another source, incorporating them into the pre-existing format. A template in Home Assistant is actually the Jinja2 templating language, which in itself is strangely named as it is more like a cross between a macro and scripting language. Templates are the closest thing Home Assistant has to a scripting language allowing you to manipulate values, make decisions, and do other scripty sort of things then use the result within automations, as a sensor value output, within entity definitions, etc. So it is very powerful and forms a key part of what Home Assistant is able to do.
It was a fair amount to digest in the beginning.
I have to laugh at myself because when I was first configuring things, for about an hour I couldn't wrap my head around how to make a simple light turn on and off. In the physical world, a "light" is actually a switch, then the bulb (the light) itself. So it seemed completely reasonable that in Home Assistant, I would define a switch, then connect that switch to a light. Which is absolutely wrong because a "switch" is just a thing humans need to turn a light on and off. Home Assistant considers a light to be something it can turn on and off by command. So I went around in circles defining dummy switches connected to nothing, creating automations to make their state changes trigger the state change of a connected light, and just generally creating a mess. Then it hit that no, a switch isn't necessary. Define a light, and Home Assistant knows how to represent it to a human in the UI, and can control it. Seems silly now that I look back on it, however the documentation doesn't really explain what a "switch" is. In reality, a switch is just a generic on/off device. So one could actually define a light as a switch and still turn it on and off, but lose access to the other attributes (such as dimming).
Upgrading |
Upgrading is big part of Home Assistant at the moment as it is in continuous development. Though very stable, it is still sub version 1 alpha software.
I consider Home Assistant to be critical infrastructure so I have a very conservative approach to upgrading. When I began with Home Assistant, the release cycle was about every two weeks. Now they have moved to a monthly release cycle. Typically I wait 3 months at least between a release and considering the upgrade so all bugs and breaking changes are well known. Unless there is an obvious change that would make a big impact. For example, in early 2020 they removed the older "States" interface, replacing it with the new "Lovelace". I was not willing to recreate my interface (it actually turned out to be fairly straightforward) so I hung out on the older version for 6 or 8 months. Before any update I review all the breaking changes introduced, then take a snapshot of the VM prior to running the upgrade. Incidentally I usually upgrade 3 versions at a time, so in reality there are times when my installation is 6 months behind.
My update procedure is fairly stress free:
In general I have had to make very few config alterations due to breaking changes. At the most it may have been just adjusting one or two parameters of an entity/integration, or moving an entity/integration from YAML config to the UI.
Taking a VM snapshot means that if anything goes truly pear shaped during the upgrade and gets into a state where Home Assistant is non-recoverable in a reasonable amount of time, the solution is to just spend a minute rolling back the snapshot.
Home Assistant's text based configuration makes it relatively painless to move to a new installation. So periodically, when there are a lot of breaking changes or when I have let the installation get many releases behind, I prep a new VM. This new VM has an up-to-date Ubuntu Server installation as well as requisite software (such as Python, nginx, MySQL) installed as their latest version. The latest Home Assistant version is then installed and the configuration files copied over. Any breaking changes are resolved. The the old VM is shut down and the new VM assigned the same IP address as the old. Thus the upgrade to a completely new installation and environment is completed with mere minutes of downtime. At some point these migrations will become unnecessary as Home Assistant matures.
Remote Access via nginx Reverse Proxy |
I discovered early on that nearly all external services which can call Home Assistant (ie. Google Home) require Home Assistant APIs to be served with a valid (non-self signed) SSL certificate. But, I prefer to access Home Assistant services internally via the internal IP address and plain HTTP (I have written some scripts which interact with the Home Assistant HTTP API). HA isn't capable of serving both HTTPS and HTTP at the same time which leads to the problem of that once HTTPS is enabled, one must access HA using HTTPS. Internally, this means either running split DNS or face certificate errors when hitting HA via internal address vs. external host name.
The solution is to serve HA externally using SSL, while internally HA continues to run on plain HTTP. The Home Assistant documentation used to have a good set of instructions for accomplishing this via nginx, so I simply followed them to configure nginx as a reverse proxy serving HA on HTTPS externally.
This information seems to have been removed from the Home Assistant website so here is the nginx site configuration I use:
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
# Update this line to be your domain
server_name example.com;
# These shouldn't need to be changed
listen [::]:80 default_server ipv6only=off;
return 301 https://$host$request_uri;
}
server {
# Update this line to be your domain
server_name example.com;
# Ensure these lines point to your SSL certificate and key
ssl_certificate /etc/nginx/ssl/my_certs_bundle.crt;
ssl_certificate_key /etc/nginx/ssl/my_certs.key;
# Ensure this line points to your dhparams file
ssl_dhparam /etc/nginx/ssl/dhparams.pem;
# These shouldn't need to be changed
listen [::]:8124 default_server ipv6only=off; # if your nginx version is >= 1.9.5 you can also add the "http2" flag here
add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
proxy_buffering off;
location / {
proxy_pass http://127.0.0.1:8123;
proxy_set_header Host $host;
proxy_redirect http:// https://;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
|
It was actually a rather straightforward process of installing nginx, then following some instructions to setup an enabled site configured as a reverse proxy using the configuration (above) supplied from the Home Assistant documentation. I obtained an SSL certificate for my Home Assistant domain name and added it to the configuration. Then simply mapped the new proxy port through the router. I think it was one of those rare circumstances that it actually worked first try. Considering it was my first ever experience with nginx, yay for me.
For many reasons I am not going to fool around with Let's Encrypt free certificates requiring services to run for periodic renewal. A real SSL certificate is under $20 yearly, comes from a trusted authority, and does not rely on a convoluted automatic renewal process every 90 days.
Mosquitto MQTT Broker |
As covered in Protocols and Standards, I have standardized on MQTT as a communications protocol between Home Assistant and devices.
Until release 0.112 Home Assistant included a built in MQTT broker which made using MQTT a no-brainer and extremely easy to set up. The built in broker was removed in 0.112 which required migration to a new broker.
I chose the Mosquitto MQTT Broker because it seems to be the most standard, widely used cross platform open source broker available. A simple
apt install mosquittoand the broker is running.
I also have it installed on Windows for development use.