Differences

This shows you the differences between two versions of the page.

--- sonoff [23.01.2018 00:33] – [Rules Engine - node-red] Pascal Suter
+++ sonoff [08.12.2018 09:57] – [Rules Engine - node-red] Pascal Suter
@@ Line 11: / Line 11: @@
 ===== alternative firmware =====
   * [[https://github.com/arendst/Sonoff-Tasmota|Tasmota]]
-    * works also on some other ESP based devices
+    * works also on lots of ESP based devices and can be easily adopted to support even more. for example [[MagicHome|MagicHome RGB LED Controllers]]
     * provides control via serial / web / MQTT
 ===== What I intend to do with it =====
@@ Line 22: / Line 22: @@
 at first i will use the existing MQTT bridge for DS which uses the Web-API. In the future i would like to write a little module that uses the Virtual Device API of digitalSTROM which would allow me to emualte a digitalSTROM device for my sonoff. this would make the integration even smoother and there would be no need for an external rules engine anymore. However, i'll get started with the web-api and rules engine first, as I'd like to try out some MQTT stuff first :)
+===== The simple solution =====
+Isn't there a simple solution to that? Shure there is, but what fun would it be? :)
+the digitalSTROM server comes with some "apps" that can be installed on it via a repository. one of the standard apps that is even pre-installed is called ''Event-Responder'' and does all we need in order to get our sonoff to turn on and off depending on scenes selected in rooms. Because it is so easy to do, I will quickly hightlight how to do all that before I explain the more complex solution:
+You can either flash the Tasmona firmware as described below or hack your sonoff in any other way with any other firmware or even with the original firmware in place. All you need to achieve is, that you can send a http-get request to the sonoff to turn the light on and off. Tasmona does that, and since I am going to use it anyway for my project, I will assume you also flashed that. Also this solution requires the sonoff to have a static IP .. either do this with a reservation in your DHCP server or enter a fixed IP in the web-interface of Tasmona.
+In the dss web menu go to apps and open the ''Event-Responder'' app. create a new event and select the room and scene you want to use as trigger for your sonoff to turn on. as initiated activity select "Call URL" and enter ''%%http://<sonoff-ip>/cm?cmnd=Power%20On%%''. then give it some useful name and save.
+Create a second Responder, but this time select all other scenes in the same room as trigger (you can do that by holding CTRL on your keyboard while clicking on all the scenes you do NOT want your light to be on). then as action use again Call URL but this time enter ''%%http://<sonoff-ip>/cm?cmnd=Power%20Off%%''.
+that's it, you're all set. Congratulations you just saved yourself a day worth of fiddling with tools and scripts :) So why am I still going down the complicated route.. well, mainly **because I can** :) and also because i want to use more devices including some input devices in the future to do more complicated stuff which exceeds the capabilities of my DSS. Also, the Event-Responder is quite slow, it is even slower than my first solution with the DSS-API based mqtt-connector. So expect to wait 2-3 seconds for your light to go on. Unless you have a newer DSS than me. The newer models (DSS20 and 22) have faster cpu's and should be able to process these responders alot quicker.
 ===== Falshing Tasmota Firmware =====
 this is straight forward, just follow the guides on the [[https://github.com/arendst/Sonoff-Tasmota|Tasmota github]] site. there is a wiki with loads of information.
@@ Line 30: / Line 42: @@
 Here are the steps it took to flash the stock sonoff (current and more detailed instructions can be found in the [[https://github.com/arendst/Sonoff-Tasmota/wiki/Arduino-IDE|Tasmota wiki]]
-  - download the latest [[https://www.arduino.cc/en/Main/Software|Arduino IDE]] and unpack it.
+  - [[setup arduino IDE for ESP8266]]
-  - create a subdirectory called ''portable''
-  - start your arduino IDE and go to ''File->preferences'' and add this to //Additional Boards Manger URLs//:  ''http://arduino.esp8266.com/stable/package_esp8266com_index.json''
-  - select OK and go to ''Tools-Boards...-Boards Manager...'' and enter ''ESP'' into the search. now install the ''esp8266'' module and close that window.
   - download the sourcecode tar.gz or zip package from the [[https://github.com/arendst/Sonoff-Tasmota/releases|Tasmota releases page]]
   - extract the contents of the ''lib'' directory into your arduino ide's ''portable/sketchbook/libraries'' directory and extract the entire ''sonoff'' directory into your arduin IDE's ''portable/sketchbook/'' directory
-  - verify your settings under Tools are like these: <code>
-Board: "Generic ESP8266 Module"  <<<<!!!!
-Flash Mode: "DOUT"  <<<<!!!!
-Flash Size: "1M (no SPIFFS)"  <<<<!!!!
-Debug Port: "Disabled"
-Debug Level: "None"
-LwIP Variant: "v2 Prebuilt (MSS=536)"
-Reset Method: "ck"
-Crystal Frequency: "26 MHz"
-Flash Frequency: "40MHz"
-Upload Using: "Serial"
-CPU Frequency: "80 MHz"
-Upload Speed: "115200"
-Port: Your COM port connected to sonoff
-</code>
   - edit ''user_config.h'' and enter your wifi name and password. all other settings can be changed later once the device is booted, but you can of course set them here if you want. this will make it much faster if you flash alot of these devices and want the same settings on all of them.
   - **make sure your sonoff is disconnected from the AC power lines***
@@ Line 118: / Line 112: @@
   npm install --save mqtt-dss-bridge
 this will create a sub-directory ''node_modules/mqtt-dss-bridge''. In there you can find the ''config.js'' file where you can configure the bridge. You can leave most of it at the default but some things need to be adjusted:
-  * ''refreshInterval'': 5000 milliseconds might be too long.. i set it down to 1000 for now which still is quite laggy when you want to turn other lights on or off in a room based on room actions. but i guess since it needs to actively poll the dss server, I'll try to slowly make my way to the right setting.
+  * ''refreshInterval'': 5000 milliseconds might be too long.. i set it down to 1000 for now which still is quite laggy when you want to turn other lights on or off in a room based on room actions. but since it needs to poll all the information via the API every time in that interval 1s is about all i can get. If i lower it furher it actually gets slower and the digitalSTROM app starts to be laggy too. Your mileage may vary depending on the size of your DS installation i guess.
   * ''url'' of the MQTT client: i left it at localhost, as this is running on the same server
   * ''url'' of the DigitalSTROM server: this is where you need to enter your DSS' IP Address. Leave the Port at 8080, as the API uses a different port than your web-interface
@@ Line 136: / Line 130: @@
 In order to turn the lights on in a room, you can publish to the topic ''set/dss/apartment/zones/Garden/groups/1/lastCalledScene'' and set the value to ''5'' or ''0''. here is an example using ''mosquitto_pub'':
   mosquitto_pub -h mqtt.psuter.ch -u sonoffs -P sonoff -t set/dss/apartment/zones/Garden/groups/1/lastCalledScene -m 5
 ===== Rules Engine - node-red =====
 Now that we got both our devices connected to the MQTT Broaker it is time to set up a Rules engine, to conenct the two together :)
-[[https://nodered.org|Node-Red]] looks like a very capable and interesting Rules engine. I have also spotted another interesting open source project called [[https://thingsboard.io/]] which seems to be interesting for monitoring stuff. not in my focus right now, but worth a mention in order to investigate it later some day :)
+[[node-red]] looks like a very capable and interesting Rules engine. I have also spotted another interesting open source project called [[https://thingsboard.io/]] which seems to be interesting for monitoring stuff. not in my focus right now, but worth a mention in order to investigate it later some day :)
 installation is again very easy through npm:
@@ Line 161: / Line 156: @@
 here is the exported flow in case you want to import that into your Node-RED editor to make sure you have the same settings: <code>
-[{"id":"72ae1fa8.3120d8","type":"mqtt in","z":"3bd0c5c5.ad27c2","name":"Arbeitszimmer Scene","topic":"dss/apartment/zones/Arbeitszimmer/groups/1/lastCalledScene","qos":"0","broker":"f0595b86.397c3","x":120,"y":200,"wires":[["80de7c5f.1a0798"]]},{"id":"2cbc4b68.51c464","type":"mqtt out","z":"3bd0c5c5.ad27c2","name":"garden light","topic":"cmnd/garden_light/power","qos":"","retain":"","broker":"f0595b86.397c3","x":550,"y":300,"wires":[]},{"id":"80de7c5f.1a0798","type":"json","z":"3bd0c5c5.ad27c2","name":"value","pretty":false,"x":110,"y":300,"wires":[["da2d017b.eb802"]]},{"id":"da2d017b.eb802","type":"switch","z":"3bd0c5c5.ad27c2","name":"turn on or off","property":"payload.$value","propertyType":"msg","rules":[{"t":"eq","v":"5","vt":"num"},{"t":"neq","v":"5","vt":"num"}],"checkall":"false","outputs":2,"x":310,"y":300,"wires":[["22f400d9.731088"],["16df239f.b6f35c"]]},{"id":"22f400d9.731088","type":"change","z":"3bd0c5c5.ad27c2","name":"ON","rules":[{"t":"set","p":"payload","pt":"msg","to":"ON","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":240,"wires":[["2cbc4b68.51c464"]]},{"id":"16df239f.b6f35c","type":"change","z":"3bd0c5c5.ad27c2","name":"OFF","rules":[{"t":"set","p":"payload","pt":"msg","to":"OFF","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":410,"y":360,"wires":[["2cbc4b68.51c464"]]},{"id":"f0595b86.397c3","type":"mqtt-broker","z":"","broker":"localhost","port":"1883","clientid":"nodered","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willPayload":"","birthTopic":"","birthQos":"0","birthPayload":""}]
+[{"id":"72ae1fa8.3120d8","type":"mqtt in","z":"3bd0c5c5.ad27c2","name":"Garden Scene","topic":"dss/apartment/zones/Garden/groups/1/lastCalledScene","qos":"0","broker":"f0595b86.397c3","x":100,"y":200,"wires":[["80de7c5f.1a0798"]]},{"id":"2cbc4b68.51c464","type":"mqtt out","z":"3bd0c5c5.ad27c2","name":"garden light","topic":"cmnd/garden_light/power","qos":"","retain":"","broker":"f0595b86.397c3","x":550,"y":300,"wires":[]},{"id":"80de7c5f.1a0798","type":"json","z":"3bd0c5c5.ad27c2","name":"value","pretty":false,"x":110,"y":300,"wires":[["da2d017b.eb802"]]},{"id":"da2d017b.eb802","type":"switch","z":"3bd0c5c5.ad27c2","name":"turn on or off","property":"payload.$value","propertyType":"msg","rules":[{"t":"eq","v":"5","vt":"num"},{"t":"neq","v":"5","vt":"num"}],"checkall":"false","outputs":2,"x":310,"y":300,"wires":[["22f400d9.731088"],["16df239f.b6f35c"]]},{"id":"22f400d9.731088","type":"change","z":"3bd0c5c5.ad27c2","name":"ON","rules":[{"t":"set","p":"payload","pt":"msg","to":"ON","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":240,"wires":[["2cbc4b68.51c464"]]},{"id":"16df239f.b6f35c","type":"change","z":"3bd0c5c5.ad27c2","name":"OFF","rules":[{"t":"set","p":"payload","pt":"msg","to":"OFF","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":410,"y":360,"wires":[["2cbc4b68.51c464"]]},{"id":"f0595b86.397c3","type":"mqtt-broker","z":"","broker":"localhost","port":"1883","clientid":"nodered","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willPayload":"","birthTopic":"","birthQos":"0","birthPayload":""}]
 </code>
 and here is what it looks like:
 {{ :node-red:flow.png |}}
+==== a word about retention ====
+it seems node-red sets the retention flag of mqtt by default. I learned this when i used it together with the mqtt-dss-connector: i created a rule that turned the light in the bathroom on. if i turned it off later (not via node-red) and then restarted the mqtt-dss-bridge the light in my bathroom went on again "for no reason" .. this was because that on command was published with the retention flag set to on (or actually not set to anything in node-red). so i set it to "false" and the problem was resolved.
+===== Making the installation permanent =====
+this might be different in your setup, but I would like to install the npm modules globally and then run them as non-root users through a systemd script to make sure they are auto-startet as soon as my system boots.
+before you start, make sure you export your ''node-red'' flow to a file or at least past it into a text editor so you have it available later to restore it. then stop ''mqtt-dss-bridge'' and ''node-red''
+the easy one first: Mosquitto. Since we used the ubuntu package, it already came with the necessary startup scripts, so a simple
+  systemctl enable mosquitto.service
+is enough :)
+now lets install our npm modules (''node-red'' and ''mqtt-dss-bridge'') globally:
+  npm install --global --save mqtt-dss-bridge
+  npm install --global --save --unsafe-perm node-red
+create the users and groups to run these tools:
+  addgroup --system mqtt-dss-bridge
+  addgroup --system node-red
+  adduser --system --no-create-home --ingroup mqtt-dss-bridge mqtt-dss-bridge
+  adduser --system --ingroup node-red node-red
+copy the ''mqtt-dss-bridge'' ''config.js'' file to ''/usr/local/lib/node_modules/mqtt-dss-bridge/config.js''
+create the startup service file for ''mqtt-dss-bridge'':
+<code>
+cat > /etc/systemd/system/mqtt-dss-bridge.service <<EOF
+[Unit]
+Description=Node.js MQTT DSS Bridge
+Requires=mosquitto.service       # starts mosquitto first when this service is started manually
+After=mosquitto.service       # starts this after mosquitto on boot
+[Service]
+ExecStart=/usr/bin/node /usr/local/lib/node_modules/mqtt-dss-bridge/index.js
+# Required on some systems
+#WorkingDirectory=/usr/local/lib/node_modules/mqtt-dss-bridge/
+Restart=always
+ # Restart service after 10 seconds if node service crashes
+RestartSec=10
+ # Output to syslog
+StandardOutput=syslog
+StandardError=syslog
+SyslogIdentifier=mqtt-dss-bridge
+User=mqtt-dss-bridge
+Group=mqtt-dss-bridge
+Environment=NODE_ENV=production
+[Install]
+WantedBy=multi-user.target
+EOF
+</code>
+create systemd service for ''node-red''<code>
+cat > /etc/systemd/system/node-red.service <<EOF
+[Unit]
+Description=Node.js Node-RED
+Requires=node-red.service       # starts node-red first when this service is started manually
+After=node-red.service       # starts this after node-red on boot
+[Service]
+ExecStart=/usr/local/bin/node-red
+# Required on some systems
+WorkingDirectory=/home/node-red
+Restart=always
+ # Restart service after 10 seconds if node service crashes
+RestartSec=10
+ # Output to syslog
+StandardOutput=syslog
+StandardError=syslog
+SyslogIdentifier=node-red
+User=node-red
+Group=node-red
+Environment=NODE_ENV=production PORT=1880
+[Install]
+WantedBy=multi-user.target
+EOF
+</code>
+create a rule for ''rsyslogd'' to log messages from ''mqtt-dss-bridge'' into a separate file:
+<code>
+cat > /etc/rsyslog.d/mqtt-dss-server.conf <<EOF
+if \$programname == 'mqtt-dss-bridge' then /var/log/mqtt-dss-bridge.log
+if \$programname == 'mqtt-dss-bridge' then ~
+EOF
+</code>
+create a rule for ''rsyslogd'' to log messages from ''node-red'' into a separate file:
+<code>
+cat > /etc/rsyslog.d/node-red.conf <<EOF
+if \$programname == 'node-red' then /var/log/node-red.log
+if \$programname == 'node-red' then ~
+EOF
+</code>
+restart ''rsyslogd'' and reload ''systemd''
+  systemctl restart rsyslog.service
+  systemctl daemon-reload
+  systemctl start mqtt-dss-bridge.service
+  systemctl start node-red.service
+check if the services are all running fine. if that is the case, enable them to autostart on boot:
+  systemctl enable mqtt-dss-bridge.service
+  systemctl enable node-red
+now restore your node-red settings
+===== Next Steps =====
+try to get ''mqtt-dss-bridge'' to use some sort of authentication. for encryption is not that important as the sonoff's probably won't have enough power to use encryption, and that's where it's needed most. (see [[https://github.com/arendst/Sonoff-Tasmota/issues/146|#146]])
 ===== Future projects =====
-eventually I'd like to write a script that uses the [[https://github.com/plan44/vdcd|plan44 vdcd]] external device API to integrate a sonoff device into a digitalSTROM setup as a Joker fixture (can be used for all available sort of fixtures like light, shade, ventilation etc.).
+eventually I'd like to write a script that uses the [[https://github.com/plan44/vdcd|plan44 vdcd]] external device API to integrate a sonoff device into a digitalSTROM setup as an emulated digitalstorm device. this will overcome the lag that is generated by querying the Web-API. we could then also create a dummy light bulb which will just publish its state to MQTT. by assigning it a different value for all of the scenes it is in, we can pass the room-state to MQTT with almost no lag. This will enable other devices like the sonoff to react immediately and not just during the next poll-interval.
+see my progress on that at [[MQTT for digitalSTROM]]