Doing everything from scratch - the journey so far

I’ve had a superficial look at the various open source and commercial programs but they all seem to have a checkered history with regards to stability and bug fixes. They all seem complex, which is presumably a side effect of having to be flexible and dealing with a wide range of hardware.
I’m reluctant to use any of the various ‘IOT ready’ switches or lights because I don’t want to end up in a situation where a company goes bust and you can no longer get replacement parts.

A couple of years ago as an experiment I put a bunch of business logic into an Arduino Mega2560 and linked that via a CAN bus to another Arduino at the other end of the house that had a bunch of temperature sensors, level sensors, a valve and a motor. I dumped the sensor data to a Debian box via MQTT for plotting and analysis using Python and matplotlib.

I learned several things from that project:

  • CAN bus is super reliable.
  • Forget the Arduino IDE. Use an external editor and only use the IDE for compiling and uploading.
  • Repeatedly updating an Arduino is a pain.
  • There are no Arduino debug tools other than ‘print’.
  • Keeping C++ H and CPP files in sync is a pain and semicolons should be outlawed.
  • Microcontroller user interfaces are really hard due to memory and CPU limitations etc. and should be avoided at all cost.
  • Matplotlib is strange and confusing, but does produce excellent quality plots that give a true picture of what is going on.
  • MQTT works surprisingly well.
  • Debian Linux is surprisingly stable.
  • Python IDEs aren’t great for large projects because the compiler doesn’t know the type of anything and therefore cannot reliably suggest code completions.
  • You don’t need fancy user interfaces.
  • You do absolutely need the ability to log system activity including including the user input so you can forensically diagnose problems.

As updating the Arduino was such as painful process I decided to port all the Arduino business logic to a program written in Kotlin running on a Windows PC and run it as a parallel system (like a simulator) using the same sensor data that the Arduino was using. For those who have never heard of Kotlin it is basically a less ugly version of Java that compiles and runs on a Java virtual machine just like Java does. It’s mainly used by Android developers but works fine on the desktop as well.

I learned that:

  • Windows is surprisingly stable providing that you stop it from updating itself and rebooting without your permission.
  • The free Intellij Kotlin IDE is awesome. The code completion minimises the amount of typing that you have to do and the background compilation means you don’t have to wait long when doing iterative development.
  • Logging data to a database instead of just using text files makes the analysis of program behaviour so much easier.
  • Hiding key variables behind setters and logging the state changes makes debugging so much easier.

So here I am at a fork in the road. Given the surprising (to me at least) stability of Linux and Windows there doesn’t seem to be much of a reason to keep complex business logic in a microcontroller. Having a desktop based system with a real debugger makes the development process much quicker.

So what next? I’m not too sure, making a monolithic controller that controls lots of things, like a PLC does, sounds like a bad idea. In a normal software development you have the luxury of a separate dev, test and live system. In the house you are living in there is only one system so I need to figure out a way of breaking the software into modules that can run independently of each other so that I can make changes without bringing the whole system down.

1 Like