App(s) for controlling Arduino/ESP locally

Wondering if anybody has knowledge of an app or apps that could be used to control something within your local wifi network OR direct to a device? I was thinking about creating a device that was NOT connected to something like HA or OH that I could set up an Android GUI/console of some sort to do direct device control. One example was my gate open/close system. I have all sorts of ideas on how to control it but was wondering about app access so I could pull up and tap a screen on my phone. I am not using HA or OH and was trying to not make it MQTT dependent since I have yet to get a single one of my devices to have a stable and reliable connection to MQTT that doesn’t require some sort of regular reset either through the server(s) or on the device itself.

I suppose it could be internet reliant but none of my devices need to be accessed from a remote location. All of them can be contacted through my local wifi.

Just spit-ballin’ for ideas

I googled “alternative to MQTT” and this came up: https://www.pubnub.com/solutions/home-automation-and-machine-signaling/ But it looks to be a paid service of some sort.

Have you tried running a Pi with Raspbian and vanilla Mosquitto MQTT broker(server)? Pi obviously needs a static IP.

Mine has been running flawlessly for almost two years now (bar a few random grid blackouts). All my end-devices have that reconnect function that sets a random number to the client string and connects via auto dhcp to prevent any networking issues.

Guess you could also host a web server on the Pi (or machine of choice) to build a GUI. Then you can use any local device with a web browser for control. But you’d still need a light-weight system for communication to your end-devices.

Personally when I get some time I will be moving all my gear over to Mozilla WebThings. It has a really nice interface and a bunch of really good arduino stuff. You can get a custom URL that handles all the SSL + tunneling so you can just add the web interface as a browser shortcut on phone and it JustWorks™ where ever you are.

I can honestly say I have NO IDEA how I ended up there but I found an incredibly interesting setup for exactly what i was looking for. Oddly enough, searching for this kind of thing netted me nothing and then I fall into it :joy: I found an app development… um, platform?.. that looks like with half a lifetime of studying it, I could probably build some seriously connected apps for not only Android but iOS as well!

EVO Things - BLE

EVO Things is not just for bluetooth low energy but it has been developed to create apps for embedded processors for things like Arduino and ESP projects. It appears to be pretty complicated since I don’t know anything about JSON scripting and am pretty behind the curve on web page development (I used to be a web designer, oddly enough, back in the mid 90’s, long before CSS existed). However, it looks like it is well supported and has a pretty good sized group involved so it isn’t flying blind and alone. The BLE part of the development is intriguing though because lots of what i am playing with could really be fun if I went to the BLE side of the tracks. The package vault could benefit nicely from it in the future, I am thinkin’!

@chris I am JUST now getting around to dropping reset code into a couple of my devices to see what kind of difference that makes. I have the Hassio plugin that is Mosquitto and HiveMQ running on a Pi and I have a VM running Mint and a Mosquitto broker running on that as well to do some testing. Doesn’t seem to make any difference which broker instance I am using, I get the same kinds of issues. So, in my troubleshooting, I am looking at network things like wifi strength (for a couple devices) and possible disconnects. I put the reconnect code into the vault code and dumped it onto my bench model of the vault controller and have a copy of MQTTbox subscribed to the beacon channels of 3 devices I restarted. I want to see if the ones with reconnect code hang in there after the others drop off the radar. I looked at MQTT-FX like @Bedrock suggested in topic 463… Arduino Mega and 8 relay shields topic

It seems like it is one hell of a toolbox item, though it has very little documentation and I can’t get the Broker Status tab to work… shows me squat for info… ever. But I am slowly honing this down to see exactly where the problem lies. Just looking for consistent failures so I can start in a less vague direction.

1 Like

i had my MQTT server running for a little over 2 months and i haven’t ran into any issues, MQTT.fx has been one hell of a tool its really handy as you can subscribe to multiple topics and even color code them for trouble shooting.

1 Like

My first arduino project was to light up a Halloween display. It had colored LED lights, some old fashioned model RR street lights and some LED “flame” lights which flickered like candle lights. It was before I decided to go the MQTT route. The unit has a push button to toggle through different lighting scenes. It also uses a web page which I pass parameters to either with a web page showing links, or via HomeSeer, which I use for most of my automation controls. I have since, however, switched to using a Raspberry Pi as an MQTT broker and have a phone app which can send topics/values via a screen of customizable inputs. I normally use that app for testing myself, but you could do the same. I am running the mosquitto broker on the RPi. While the web page hosted on the arduino works well, I just found the MQTT approach to be easier to use with the home automation system.

Hey there @Karl_S

Yeah… I am basically in the same place. I have an RPi as a mosquitto broker and have my devices communicating with that and I use a couple different MQTT apps for sending messages and what not. I was fishing for an app that allows you to use a library or code set that would do direct communication with a device, like through BLE or local wifi. So a direct access thing, not like through a broker… MQTT IFTTT, Etc. I don’t know that it exists.

For most everything I could come up with for my personal automation projects, the MQTT enabled setup is fine. I can use NodeRed to create dashboard type pages with inject nodes to control anything, really. But I am trying to solve a “closed loop” security problem. My package vault project is open source and that creates some interesting problems when it comes to security. You can’t broadcast the security code to your alarm system and expect thieves to not notice. So I am trying to integrate smart IoT tech into an open source project and I am discovering that the only IoT stuff that has any kind of security aspects, has dedicated proprietary IoT controls… like through a Chinese server somewhere. I would like, though I doubt it is possible, to have some IoT smarts that could be programmed into an app that works line of sight (BLE) or fenced in (direct local wifi) that could be shared as part of the open source project but wouldn’t be something someone could easily just clone a remote for and just bypass your security and access your box. The current plan is to use a keypad or HID security RFID access for unlocking the box because it is closed loop. The only thing more closed loop is a key… and that is exactly what i am trying NOT to do.

I pondered using an RPi as a brain that runs a Mosquitto broker, NodeRed and a dashboard then you could access the box controls through some sort of webpage to change keypad access numbers and set timers for locking and all sorts of stuff but then I am in the embedded computer world and I can’t possibly support that, even for a close friend, because I don’t know the OS and apps well enough to assist anyone with fixing it when it WILL break. Just because it can be done doesn’t mean it is a good idea and I don’t want to push anyone down a path that is going to just be annoying. We all know how troublesome this IoT world can be and we are working on our own crap, not others.

1 Like

@Guru_Of_Nothing If you have all of your controls behind your firewall, password protected, you should be fine provided nothing needs to go outside your network for any reason.

HomeSeer has a Raspberry Pi option but there is a cost and is limited to, I think 5 Linux plug-ins. It can run completely local though, requiring no external internet access. If you want phone app access you can create custom touch capable screens but that requires an additional application. You can also access the web page you would use at home and can password protect it as well They have a free phone app which automatically makes buttons for you but I think that requires connection to their web server, which I believe is in the states. I am fairly certain the HSTouch allows you to stay local or can be accessed across the web if you expose it and has the ability to allow for multiple users with different permissions, all password protected. So without needing to write a lot of code it provides a quick hub to control a lot of items. I run it on an old Dell pizza box desktop. Doesn’t require much as far as a system is concerned but with the Windows version I can have many plugins for things like calendar integration, IP cameras, MQTT, control of my A/V equipment, and more.

I cannot speak to NodeRed, Home Assistant, or others. Most of these were not around when I started down this rabbit hole 14 years ago. I know many use some of the open source hub applications but I do not know if they require internet access for anything.

Your MQTT Broker should be a local network item since it is on a RPi. So there should be no fear of it being hijacked unless someone gets onto your network. Note that the RPi running the broker could be in your house and you only need one broker. It is the switchboard operator for MQTT so it just receives and distributes the topics. You could have your arduino in your box subscribe to topics. And if you use different topics on your system than what you post in your code, and also use different IPs than those in your posted code, you are not giving away the keys. Sure, if I could get on your network I could try to find the MQTT broker and then use an app to listen to the topics, but I need to be on when a topic is sent. As for setting dummy settings in your code and instructing others to change it to suit their needs, people do that all the time so anyone coming behind you should have little problem doing this provided you add comments/instructions to do so. In most cases these are constants at the beginning of the code. Bottom line: Don’t confuse the Mosquitto broker with the cloud based brokers out there. It does not require an internet connection to do anything and is therefore as secure as your network. Shoot! It could all be hardwired and on a part of your network not accessible by wifi if you wanted. Especially if you are building any of the MQTT units.

Take a look at what this guy has done: MQTT Wall Controller - project update - prototype in the wall. As I recall he has the ability to send the arduino MQTT codes to change the operations of the buttons on his wall controller. You could probably do something similar to send a new security code. Of course, you could also just have access to the USB on the arduino and connect a laptop to the unit and download new code with an updated security code. That would be a lot less programming and since you can mechanically unlock your box, you should be good. And speaking of that, your biggest security flaw is that I could get to your arduino unit and re-program it allowing me access without you knowing. The only way to stop that is to run ethernet to the unit and put it in a locked metal box inside your delivery box.

Frankly, for your box (I was watching your videos earlier, nice job!) I would be willing to use MQTT to send commands to the unit provided I used a local broker (your RPi running Mosquitto) and didn’t have an open network. You could have other arduino units in the house or wherever with nothing more than a simple button that sends an MQTT topic and payload to unlock your box. Again, unless I am on your network I cannot send the same information to open it.

All this said, I also prefer to not use wifi for controlling my more important items. I especially don’t like it for something sending data around with any regularity. That is why much of my stuff is z-wave. It is off my data network. But MQTT is very light and fast, which is why I have moved to is over using a web page to control my arduinos.

I hope this was helpful. My apologies if I repeated anything you already understand.

And the comment “Just because it can be done doesn’t mean it is a good idea…” is something I say all the time. And I often enforce it with “I can put my car in reverse, but I don’t drive to work that way.” :-):smiley:

Karl

Oh yeah! For my own vault, I use MQTT and it is all within my network and that is firewalled from the world with a PfSense box and what not so… let 'er buck! But it all became something different when I started getting requests from people to create one for them. Makers like us create the infrastructure we need for our hardware and we learn it so we can support/fix it when it crashes. A little built in complexity is not a problem.

But I got a request for a box and they asked of they could have it with the bells and whistles I am integrating into my box… internal camera, wifi messages that a package has arrived, remote (like pushbutton open from location other than at the box), keypad with changeable codes, etc. And I had to scratch my head and go…uh… No. Why? Because I have an external computer that does all the external connectivity and I don’t linux well enough to even be slightly comfortable with trying to set one up for them. And the stuff that allows easy parameter changes requires additional stuff… like my laptop with Arduino IDE and all the current libraries used, so i can plug in and reflash, SSH access to the Pi to restart services or add apps or update existing apps. And even though OTA sounds like a great idea, when s*&% hits the fan and you brick it, you are back to the beginning so any kind of update is a problem.

So I had a “return to reality” moment where I had to decide what was truly possible in an open source environment for a ‘non-maker’ to have what I am trying to create. It has to be closed loop. It has to be easily update/repair/replace-able if something went wrong like the controller craps out. Can I send them a controller they just plug in and it works again or do they have to download a bunch of software, update libraries, parse code, flash chips, get IFTTT registrations, and who knows what the hell extra? Which is the cruxt of creating an IoT device. Is it like a coffeemaker that dies and gets thrown away or is it a module that gets replaced that has all the brains to the operation to fix it and it is proprietarily driven… or is it open-source and possibly hackable and requires tons of auxiliary supporting applications and…

If a simple phone app allowed you to do programming and interact with the board, then great. But I only know of apps that require other supporting hardware like an RPi with linux and other software, that can be unstable and really hard to debug… and could easily require me to spend hundreds of hours doing phone support on.

I could use mqtt.org’s “test” server but that is frowned upon for commercial purposes. I could use IFTTT but they have to set it up and that is another level of complexity I don’t want to get started. It all requires external servers and like Jibo and many other businesses that use outside systems to make their crap work, when the system breaks down, you own expensive, unserviceable junk. In-house ecosystems are the ONLY way to go in my opinion and I am failing at coming up with something that can be openly shared that has the simplicity of a plug and play device with the security that this device requires for proper operation.

Ah. I understand now.

If I were doing your requested box and not looking to mass produce the thing, I would make it completely stand alone and have it use a self hosted web page. As long as they are on their network, everything would work fine for them. You could assign an IP to the unit in its code and give them the IP as the web address. They could use their phone to view the web page or a computer, but only while on the same network. Email functionality COULD be possible with messages sent/received that way, but I don’t know I would want to write that code. The main page can have all the buttons and show the latest X photos, which are saved to an SD card. If you wanted to you could set them up to allow access from outside the home by having them serve the web page from the external IP but that changes with some power outages for most and steps outside of plug and play, normal person setup, It is a lot of work and, if connected via wireless, the metal vault starts to limit your range. But the reality is that it should last a while and there should be no need to update the system. Make the arduino unit removable and you could replace it easy enough if needed with a new unit.

If you go the route of using a web page you could allow for an entry of the number of deliveries expected when grater than 1. This would mean setting a number the morning or day before the deliveries and increment an open count for each time your time-out for the possible rain deliveries occurs.

Maybe you can go with a simplified version which only allows for a manual unlock and is reset internally? That becomes no IoT, no app/web site, etc. Not as fancy but it is the easiest.

Karl

The web page integration piece is something I am interested in but is outside my wheelhouse at this point. I understand just enough of the basics to create something that will fail more than it works. So until I have the opportunity to create such a beast, I am leaving it as totally self contained. The code makes the box work as designed 100% of the time. If it stops working, the troubleshooting is simple. I already have that functioning on my prototype box. I am adding HID RFID card access for unlocking and that is a simple integration (though now that I said it, it is going to fight me :slight_smile: It will work and can be opened with half a dozen programmed RFID cards and that is is simple as it gets. If the “controller” fails, I can mail them a new one to just plug and play and they are up and running again. I am going to add a couple different LED blinks for diagnostic purposes to make troubleshooting easier should there be a need. Down the road I may be able to offer an upgrade and I hope to be able to add in some extra connections for that possibility. But who knows. The main point was just to have a super secure place to have large packages delivered. Then it got all sideways with what it might be able to do and I had to reign it in a bit. I will never mass produce them as I don’t have that kind of time or facilities. Just a hobby item that anyone with the inclination can recreate with free designs or I can create one for them for a small fee.

1 Like