Published January 29, 2020 © CC BY

Using Kniwwelino with Node-RED and your own MQTT Broker

This project shows you how to use a custom MQTT broker and Node-RED with Kniwwelino to build a YouTube subscription counter.

BeginnerFull instructions provided1.5 hours568

Using Kniwwelino with Node-RED and your own MQTT Broker

Things used in this project

Hardware components

Kniwwelino

Raspberry Pi 3 Model B

Software apps and online services

Node-RED

Mosquitto

Story

Idea

It's less about building a YouTube counter, but more about how to use an own MQTT broker with Kniwwelino. By default all the MQTT messages are send to the MQTT broker of Kniwwelino in a kind of sandbox. All topics for messages from users are normally prefixed by a kniwwelino/. You can define your own prefix-topic which is put right after the kniwwelino/, but you are still sending your messages to the sandbox. With the "Use MQTT broker at host"-Block the possibility to use your own broker is introduced to the Visual Programming Platform of Kniwwelino. You have now the possibility to install a Raspbery Pi with Mosquitto and Node-RED to keep your messages in-house and enables you to connect your Kniwwelino to the hole world of IoT.

As an easy example we build here a simple YouTube subscription counter. The data displayed on the Kniwwelino's LED matrix is harvested by a simple Node-RED flow from YouTube. As well the button events for button A, B and the combination of buttan A and B are send as well to the topic youtube/button/#.

Prepare the Raspberry Pi

First you will need a Raspberry Pi with a SD card loaded with Raspbian. You can create it easily prepare it with Balena Etcher. Create the a file called ssh in the boot partition for your SD card, to enable the ssh server on your Raspberry. Connect it by cable or WiFi to your home network.

Installing Mosquitto as MQTT broker on Raspberry Pi is quiet simple:

sudo apt-get install mosquitto

I will not go into details about installing Node-RED on Raspberry Pi, as it is already there in the Desktop version of Raspbian. And for installing it from scratch you should read the official instructions.

As a summary:

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)
sudo systemctl enable nodered.service
sudo reboot

You can access your Node-RED instance now by an other PC in your network via the browser on the URL IP-of-Raspberry:1880

Program the Kniwwelino

With the Kniwwelino sketch that you can open the Visual Programming Platform, you can program the basic functionality to send and receive MQTT messages to your broker. Just replace in the broker block the IP by the one of your Raspberry. Username and password can be left as they are on a standard installation of mosquitto.

Simple Kniwwelino Sketch to use own MQTT broker

Import the flow attached to this write-up into your Node-RED then you will get 2 short flows. The pink MQTT nodes will have red dot after import, that because you have to configure your broker first. The first flow is to receive everything from the Kniwwelino.

receiving all messages in the youtube topic and sending it to a debug node

After deploying the flow you should see in the debug messages something like this after pressing a button on the Kniwwelino:

debug output

The second part of the flow, does the magic. It requests every 5 minutes the YouTube channel of Kniwwelino and extracts the contend of the span with the class="yt-subscription-button-subscriber-count-branded-horizontal subscribed yt-uix-tooltip" you need to escape the spaces in the class by \ to use several classes. Finally the MQTT node sends the extracted number to the LED matrix of the Kniwwlino.

extract something from a website

Now you can tinker around with the incoming messages from the buttons and the possibilities of Node-RED... ;-)

Build an enclosure

I used the lower part of an old iPad box to build a "picture frame" for the YouTube counter.

To power the Kniwwelino I added Micro-USB breakout board in the back of the box and fixed it with a substantial blob of hot glue.

To make the front nice I printed the logos on a nice sheet of photo inkjet paper and laminated it with self adhesive film. To mount the Kniwwelino I glues a Kniwwelino shell on it.

After bringing the cables from the Micro-USB breakout to the front. I soldered them to the battery connector and snapped the Kniwwelino into the shell.

Finally done

We know there is not much now in the YouTube channel of Kniwwelino, but if you subscribe we will see it within 5 minutes ;-)

[
    {
        "id": "8b734ea8.52eea",
        "type": "mqtt in",
        "z": "8a3ae97e.aee128",
        "name": "",
        "topic": "youtube/#",
        "qos": "2",
        "datatype": "auto",
        "broker": "94fcb17f.c439a",
        "x": 360,
        "y": 100,
        "wires": [
            [
                "e9fd5ba9.f73398"
            ]
        ]
    },
    {
        "id": "e9fd5ba9.f73398",
        "type": "debug",
        "z": "8a3ae97e.aee128",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "x": 570,
        "y": 100,
        "wires": []
    },
    {
        "id": "ae42d176.b91b7",
        "type": "http request",
        "z": "8a3ae97e.aee128",
        "name": "",
        "method": "GET",
        "ret": "txt",
        "paytoqs": false,
        "url": "https://www.youtube.com/channel/UCZwwe6NYPhCnPdZ6Unls7-g",
        "tls": "",
        "persist": false,
        "proxy": "",
        "authType": "",
        "x": 1300,
        "y": 100,
        "wires": [
            [
                "dbf7a66a.1e97f8"
            ]
        ]
    },
    {
        "id": "3597b863.220bf8",
        "type": "inject",
        "z": "8a3ae97e.aee128",
        "name": "",
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "repeat": "",
        "crontab": "*/5 6-15 * * 1,2,3,4,5",
        "once": true,
        "onceDelay": 0.1,
        "x": 1130,
        "y": 100,
        "wires": [
            [
                "ae42d176.b91b7"
            ]
        ]
    },
    {
        "id": "dbf7a66a.1e97f8",
        "type": "html",
        "z": "8a3ae97e.aee128",
        "name": "subscribers",
        "property": "payload",
        "outproperty": "payload",
        "tag": "span[class=yt-subscription-button-subscriber-count-branded-horizontal\\ subscribed\\ yt-uix-tooltip]",
        "ret": "html",
        "as": "multi",
        "x": 1470,
        "y": 100,
        "wires": [
            [
                "57b7c6ae.62e488",
                "b56b7d37.f5bce"
            ]
        ]
    },
    {
        "id": "57b7c6ae.62e488",
        "type": "debug",
        "z": "8a3ae97e.aee128",
        "name": "",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "x": 1710,
        "y": 60,
        "wires": []
    },
    {
        "id": "b56b7d37.f5bce",
        "type": "mqtt out",
        "z": "8a3ae97e.aee128",
        "name": "",
        "topic": "youtube/MATRIX/TEXT",
        "qos": "",
        "retain": "",
        "broker": "94fcb17f.c439a",
        "x": 1780,
        "y": 120,
        "wires": []
    },
    {
        "id": "94fcb17f.c439a",
        "type": "mqtt-broker",
        "z": "",
        "name": "",
        "broker": "localhost",
        "port": "1883",
        "clientid": "",
        "usetls": false,
        "compatmode": false,
        "keepalive": "60",
        "cleansession": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "closeTopic": "",
        "closeQos": "0",
        "closePayload": "",
        "willTopic": "",
        "willQos": "0",
        "willPayload": ""
    }
]

<xml xmlns="http://www.w3.org/1999/xhtml">
  <block type="kniwwelino_MQTTsetGroup" id="iwIS@yT`|ZWom7`LKq(l" x="110" y="57">
    <field name="GROUP">youtube</field>
  </block>
  <block type="kniwwelino_MQTTsetUserBroker" id="PXJxJhCk*Wj/4/+,,,.i" x="111" y="88">
    <field name="BROKER">192.168.0.111</field>
    <field name="PORT">1883</field>
    <field name="USER">myUser</field>
    <field name="PASSWORD">myPassword</field>
  </block>
  <block type="kniwwelino_MQTTconnectRGB" id="Xy7xmV9fR=`[!YCMUBUP" x="113" y="121"></block>
  <block type="kniwwelino_MQTTconnectMATRIX" id="mI*9w.~g/6[J0E~!)(T[" x="384" y="123"></block>
  <block type="arduino_functions_setup" id="?@}vWpL7F3LVasAakMrx" x="26" y="259">
    <statement name="SETUP_FUNC">
      <block type="kniwwelino_MATRIXdrawIcon" id="z|O_2XS,QToP3}nuK4[8">
        <value name="ICON">
          <block type="kniwwelino_MATRIXdrawIconCreator" id="(B?I:UH3jE.sO8mzkn]^">
            <field name="1_1">FALSE</field>
            <field name="1_2">FALSE</field>
            <field name="1_3">FALSE</field>
            <field name="1_4">FALSE</field>
            <field name="1_5">FALSE</field>
            <field name="2_1">FALSE</field>
            <field name="2_2">TRUE</field>
            <field name="2_3">FALSE</field>
            <field name="2_4">TRUE</field>
            <field name="2_5">FALSE</field>
            <field name="3_1">FALSE</field>
            <field name="3_2">FALSE</field>
            <field name="3_3">FALSE</field>
            <field name="3_4">FALSE</field>
            <field name="3_5">FALSE</field>
            <field name="4_1">TRUE</field>
            <field name="4_2">FALSE</field>
            <field name="4_3">FALSE</field>
            <field name="4_4">FALSE</field>
            <field name="4_5">TRUE</field>
            <field name="5_1">FALSE</field>
            <field name="5_2">TRUE</field>
            <field name="5_3">TRUE</field>
            <field name="5_4">TRUE</field>
            <field name="5_5">FALSE</field>
          </block>
        </value>
      </block>
    </statement>
  </block>
  <block type="arduino_functions_loop" id="PXoI52Z{V}_)%#~m7{Lo" x="424" y="252">
    <statement name="LOOP_FUNC">
      <block type="controls_if" id="wc@o%_nsuENTf4)/pAPe">
        <mutation elseif="2"></mutation>
        <value name="IF0">
          <block type="kniwwelino_BUTTONclicked" id="iq!g)q+3eJ:K7?9#r)65">
            <field name="BUTTON">AandB</field>
          </block>
        </value>
        <statement name="DO0">
          <block type="kniwwelino_MQTTpublish" id="?Xzg/j[4HgfqxsE4u)Rt">
            <value name="MESSAGE">
              <shadow type="text" id="SkIDcl~mHRiB6qT6mw5_">
                <field name="TEXT">1</field>
              </shadow>
            </value>
            <value name="TOPIC">
              <shadow type="text" id="hV~@s,mZt7VbHKon29qw">
                <field name="TEXT">button/3</field>
              </shadow>
            </value>
            <next>
              <block type="kniwwelino_MATRIXwrite" id="cnwUqIPVC.C9rI14SBav">
                <field name="TYPE">ONCE</field>
                <value name="TEXT">
                  <shadow type="text" id="}m%1^AT?`]1t6(mLa*K5">
                    <field name="TEXT">Hello Kniwwelino</field>
                  </shadow>
                  <block type="kniwwelino_getSSID" id="8:{w{zSg-R}i[;z`E-VT"></block>
                </value>
              </block>
            </next>
          </block>
        </statement>
        <value name="IF1">
          <block type="kniwwelino_BUTTONclicked" id="kHV9GV|pOFw0lqJO!kw?">
            <field name="BUTTON">A</field>
          </block>
        </value>
        <statement name="DO1">
          <block type="kniwwelino_MQTTpublish" id="|UGo}K2j/jZwC`qx`WR{">
            <value name="MESSAGE">
              <shadow type="text" id="|a5xyVM-PTex-SlzpH4b">
                <field name="TEXT">1</field>
              </shadow>
            </value>
            <value name="TOPIC">
              <shadow type="text" id="MXNQStofPd*{E|SGS*p.">
                <field name="TEXT">button/1</field>
              </shadow>
            </value>
          </block>
        </statement>
        <value name="IF2">
          <block type="kniwwelino_BUTTONclicked" id="eGBGz/(y-jLKkr603ll_">
            <field name="BUTTON">B</field>
          </block>
        </value>
        <statement name="DO2">
          <block type="kniwwelino_MQTTpublish" id="h2veRYxXp-k#5OTpw}iT">
            <value name="MESSAGE">
              <shadow type="text" id="@F[[02xt@IRJss_gcrH!">
                <field name="TEXT">1</field>
              </shadow>
            </value>
            <value name="TOPIC">
              <shadow type="text" id="[^Catu*3O;9Cvz+8H@2O">
                <field name="TEXT">button/2</field>
              </shadow>
            </value>
          </block>
        </statement>
      </block>
    </statement>
  </block>
</xml>

Using Kniwwelino with Node-RED and your own MQTT Broker

Things used in this project

Hardware components

Software apps and online services

Story

Idea

Prepare the Raspberry Pi

Program the Kniwwelino

Build an enclosure

Finally done

Code

Node-RED flow to show MQTT messages and extract youtube channel subscriptions

Kniwwelino YouTube counter

Credits

Christian Moll

Comments

Embed the widget on your own site

Using Kniwwelino with Node-RED and your own MQTT Broker

Using Kniwwelino with Node-RED and your own MQTT Broker

Things used in this project

Hardware components

Software apps and online services

Story

Idea

Prepare the Raspberry Pi

Program the Kniwwelino

Build an enclosure

Finally done

Code

Node-RED flow to show MQTT messages and extract youtube channel subscriptions

Kniwwelino YouTube counter

Credits

Christian Moll

Comments

Related channels and tags