Thursday, February 11, 2016

Sending IoT sensor data to Hawkular-Metrics via MQTT

The other day I was writing about 'RHQ-Metrics and Grafana' and was describing how you can incorporate data from other system management agents.

Fast forward a bit and Raider is now called Twix and RHQ-Metrics has morphed to Hawkular-Metrics under the Hawkular.org umbrella.

Recently I have also been playing with Arduino and Co. and got myself also an Adafruit Huzzah ESP8266 board. This is a breakout board with the ESP8266 microprocessor on, that has a bunch of IO pins and built-in WiFi. With the default firmware it is programmed in Lua.

Huzzah on Breakout board
Huzzah on breakout board

While one can program the ESP from the Arduino IDE, I thought to give Lua a try (also to get a feel for the difference to the WiPy, that also comes with Python as a high level language). What is nice with the ESP and the NodeMCU firmware is that it already comes with support for networking, 1-wire, MQTT and more out of the box.

To get started I took the hello-world of IoT-sensors and hooked up a DS18B20 OneWire thermo sensor (for those old enough, I did that in the past with RHQ) up to the ESP and then have this communicate to a MQTT Broker (mosquitto).

Hawkular Metrics IOT
Setup diagram

As said before we have with Ptrans a universal protocol translator that can be used to feed data from collected, ganglia and others into Hawkular-Metrics. I've taken that and added support for MQTT (in my personal repo for now). Ptrans will now connect to a broker and listen on the '/hawkular/metrics' topic for data that needs to be in graphite format like

path value [timestamp_in_s]

The timestamp is optional, as in my case I was not able to get any real time clock data from the micro controller (there seem to be variants that have a clock on board).

To see the data that is coming from the device I can just run

$ mosquitto_sub  -t /hawkular/+
16617927:40.176.91.120.5.0.0.125 24.625
16617927:40.176.91.120.5.0.0.125 23.9375
16617927:40.176.91.120.5.0.0.125 23.8750

So here NodMCU with ID 16617927 and thermo sensor 40.176.91.120.5.0.0.125 is reporting around 24 deg Celsius.

I will post more on the topic in a laster posting.

References:

Tuesday, January 05, 2016

Arduino powered X-Mas tree

In 2014 we got two cats and decided not to have a classic christmas tree. I also started playing with Arduino and other gadgets, so this made me think to have an Arduino powered tree in 2015. In November my daughter and I started to build the tree.

The tree

The tree has 14 RGB LEDs (basically WS2812 ones) from Sparkfun (or EXP-Tech).

Pinout of the LEDs is as follows:

LED Pinout
LED-Pinout (click to enlarge)
.

With that, the tree wiring got relatively easy by bending the LED pins in the four different directions and then connecting all 5 Volt pins with the red wire, GND with the black and chaining data in and out either by directly soldering the pins or via the yellow wire:

Backside of the tree

Detail view:

Wiring detail

The overall schematic then looks like this (I am using an Arduino at the moment, but will use a TrinketPro in the future):

Weihnachtsbaum Schaltplan
Schematic (click to enlarge)

I've added a photo resistor and a white diode to provide some lighting of the tree front, as at night, the RGB LEDs are so light, that the tree itself can't be seen. The resistor value also serves to dim the RGB LEDs at night.

The software driving the LEDs is basically the Strandtest one that Adafruit delivers as example for their Neopixel Library. I've modified the code a bit to also show other patterns.

In the future we will add a stand for the tree that will also house the electronics and will also host some additional surprise that I will talk about in a future post :)

Tuesday, October 27, 2015

WiPy on the home network

Recently there was a Kickstarter about "an Arduino that runs Python", the WiPy. This is a small IoT board with WiFi on it (which is not available on stock Arduinos) for an attractive price. And being able to use a high level language along with a lot of existing libraries makes it attractive too.

A few days ago I got the WiPys that I backed and the first obstacle to get the running was the power supply as they do not have a (micro) USB connector on board (and I also did not back the extension boards they offered).

Luckily I still had an old defunct USB hub from which I could solder out the connector and put it on a small PCB for this purpose (while doing that exercise I also found out that the 5V of the USB port are allowed to be 4.75-5.25V and on the end of a hub even being as low as 4.4V).

WiPy on BreadBoard
WiPy on Breadboard with USB power supply connector.

Now that the WiPy is on the breadboard, I set up my computer to scan for the WLAN of the WiPy and then telneted into the device. I poked around and tried to inspect the WLAN settings as described in the manual:

>>> from network import WLAN
>>> wlan = WLAN() # we call the constructor without params

This immediately made the connection drop. It turned out I needed to first update the firmware to the latest version (v1.1 at time of the writing), which was painless (but a bit confusing, as they also supply a bootloader.bin, that is not needed).

After the firmware upgrade the above worked and I tried to change the device to talk to my home network. The caveat here is that at the moment you issue

wlan.init(WLAN.STA)

to put the machine into station mode (default is being an access point), the connection drops. Some users have solved that by connecting via UART and serial line, but I did not really want to go that route.

Instead I edited boot.py on my local computer and then uploaded it via ftp into /flash. I found out, that if you don't terminate the ftp client and have a telnet connection open as well, I could easily (syntax) check the uploaded file by pressing Ctrl-D in the terminal:

MicroPython v1.5-1-ge954604 on 2015-10-21; WiPy with CC3200
Type "help()" for more information.
>>>    <control-D>   <-- here
PYB: soft reboot
Traceback (most recent call last):
  File "boot.py", line 17, in 
NameError: name not defined

The WiPy tells me that there is an error in my file, so I edit it locally and upload it again via the ftp connection. And only at the end when the WiPy is happy, I press the hard-reset button.

Now for reference my boot.by that worked for me:

from network import WLAN

SSID = 'home_SSID'         # SSID of your home network
AUTH = (WLAN.WPA2, 'very_secret') # WPA secret as 2nd param
IP = '10.1.2.42'           # Fixed IP the device should get
ROUTER = '10.1.2.3'        # IP of your router
DNS = '10.1.2.3'           # IP of your DNS server
NETMASK = '255.255.255.0'  # Netmask to use

import machine

wlan = WLAN()

if machine.reset_cause() != machine.SOFT_RESET:
    wlan.init(WLAN.STA)
    # configuration below MUST match your home router settings!!
    wlan.ifconfig(config=(IP, NETMASK, ROUTER, DNS))

if not wlan.isconnected():
   wlan.connect(ssid=SSID, auth=AUTH, timeout=5000)
   while not wlan.isconnected():
      machine.idle() # save power while waiting
   print('WLAN connection succeeded!')
Parts of that script were taken from the WiPy WLAN tutorial and this WiPy Forum post.

Tuesday, October 06, 2015

Driving a Servo on Arduino from a remote Arduino over secured radio

In Stuttgart, there is now a Hackergarden Meetup group that tinkers with whatever the people showing up want to tinker with.

In the 1st edition I was doing some hacking on Arduino where two Arduinos were transmitting data over an encrypted radio via a RFM69 radio chip. This setup is described on the Codecentric blog.

Now in the 2nd edition we wanted to build on this and control a servo motor on one Arduino remote from the other one. This video shows the end result:

On the left you see a potentiometer, that is read out and the value is then shown on the Neopixel ring. The value is also transmitted via RFM69 chip to the other Arduino that has the receiver and which then drives the servo.

The setup on the server side looks like this:

Poti data sender Steckplatine

On the receiver side we used a "normal" setup like the one described on the before mentioned report. We had an issue for a while, as the servo was on a port that was also used by the RFM69 code, but once we fixed that, it worked.

The client side code is in my GitHub fork of the Hackergarden repository - I hope it will be merged soon :)

Sunday, August 09, 2015

mBot and Lego

Recently I got my Kickstarter-backed mBots from Makeblock, some cute little robots with basically an Arduino-brain.

And as the Kickstarter campaign was successful, I also got one of the nice LED-Matrix displays with it.

Now I wanted to mount the display on the front like they show on some pictures and the range sensor as well. Obviously that does not work as they go into the same place. Unfortunately there was no additional mounting bracket supplied.

Next try was to mount it at the back, but there are no mounting holes with (or without) thread available. Luckily we have a larger Lego collection and the website claims The chassis is compatible with Lego&Makeblock parts. Unfortunately this is not entirely true, as the holes in the mBot chassis are just a bit too small for the normal lego connector pegs or axles.

But don't fear :-)

First, with some M4 screws and nuts it is easy to just mount lego parts with the help of those:

IMG 20150809 125521

And luckily Lego also supports being mounted with M3 screws and nuts, so I basically created an adapter with some Lego Technic parts:

IMG 20150809 145058~2

And finally I was able to mount the display at the back of my mBot:

IMG 20150809 151116~2

I saw an image on the forums where someone mounted the display (for my gusto upside down) and the range sensor on top, but I think this way the sensor may be too high up and not find all obstacles.

Tuesday, July 07, 2015

Running OkHttpClient from within WildFly 9 (subsystem)

A few days ago WildFly 9 was released and one of the highlight for sure is the support of HTTP/2.0 in the Undertow web subsystem. As Hawkular has recently moved to use WildFly 9 (from 8.2) as its underlying server, it was sort of natural to try to use http2 for connections from the Hawkular-Wildfly-Monitor client to the server.

One peculiarity here is that in my case the monitor client is running inside the Hawkular server, but at the end it does not matter if it is running inside a standalone WildFly server or inside the Hawkular server.

The setup

Greg Autric has written a blog post, that shows how to set up Http2 in WildFly with the offline CLI, which also works well in the Hawkular case. As the question came up: that setup also includes the https setup inside WildFly.

The only thing that is a bit problematic in the post is that setting JAVA_OPTS before starting the server will ignore all the settings from standalone.conf, which in the current Hawkular version will prevent a correct start of the bus broker (because the IPv4Only flag is lost).

So in my opinion it is better to modify standalone.conf to *add* those options to the other options that are already there:

  JAVA_OPTS="-Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true"
  JAVA_OPTS="$JAVA_OPTS -Xbootclasspath/p:/opt/hawkular-1.0.0.Alpha3-SNAPSHOT/alpn-boot-8.1.3.v20150130.jar"
  JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"

Now when I start the Hawkular server and try to connect with FireFox on the https port, I get the warning about the self signed certificate, but can connect and get the UI over a Http2 connection as described in Greg's post.

Running the OkHttpClient

As said before, the WildFly monitor client is a subsystem inside the WildFly server. I wrote a bit of client code, that is running in the subsystem (shortened):

   OkHttpClient httpClient;
   httpClient = new OkHttpClient();

   // DO NOT USE IN PRODUCTION, allow all hostnames
   httpClient.setHostnameVerifier(new NullHostNameVerifier());

   setKeystore(httpClient); // Use custom ssl factory

   String uri = "https://...:8443/";

   Request request = new Request.Builder()
            .url(uri)
            .addHeader("Accept", "application/json")
            .get()
            .build();

   // sync execution just for the post
   Response resp = httpClient.newCall(request).execute();
   System.out.println(resp.toString());

Fail?

This code works well except for the fact that is always uses Http(s)/1.1 and never Http2 as you can see from the output of the last println statement:

  Response{protocol=http/1.1, code=204, message=....}
I was playing around with various options up to a point where I thought, I have to extract the code to a standalone Java SE class to better debug it in isolation.

I wrote the class, set the bootclasspath, ran it and it worked perfectly:

  Response{protocol=h2, code=204, message=....}

So what is the difference? I removed the bootclasspath setting for ALPN, reran and the connection fell back to http/1.1.

Which is kinda strange as my client subsystem is running inside the very same WilFly server, that is running Undertow and which is able to serve http2 requests and where I added the ALPN classes through JAVA_OPTS earlier.

Now remember that WildFly is using their own classloader system (jboss-modules), which is pretty powerful in isolating deployments and classes and restricting their visibility and/or leakage into areas where they should (not) be seen.

And this in fact is what happened here as well.

Success!

So I had to explicitly add the ALPN classes to my module.xml file for the monitoring client subsystem:

  <module xmlns="urn:jboss:module:1.3" name="${moduleName}">
    <resources>
      <resource-root path="clients-common.jar"/>
      [...]
      <resource-root path="okhttp.jar"/>
      <resource-root path="okio.jar"/>
    </resources>
    <dependencies>
      <!-- modules required by any subsystem -->
      <module name="javax.api"/>
      [...]
      <system export="true">
        <paths>
          <!-- Needed for HTTP2 and SPDY support-->
          <path name="org/eclipse/jetty/alpn"/>
        </paths>
      </system>
    </dependencies>
  </module>

From the above snippet, you can see that the okhttp and okio jars are packaged in the module and are made available to my client code as well.

Now that the module.xml has been adjusted, as is well and my subsystem is using Http2 :-)

Wednesday, May 13, 2015

Scripts with multiple actions in Textual5

Some may know Textual as a powerful IRC client on the Mac.

One of the interesting parts (to me) is that Textual allows to extend it via Scripts, that can be written in AppleScript or other scripting languages.

They have an introuction to write Scripts, which I've used to create some simple scripts.

What I always wanted to do it to have scripts that can do multiple things like


/away Gone fishing
/msg I'am out of here
/leave

To set myself into away mode, put a message in the channel and then leave it.

Unfortunately, this seemed impossible.

Yesterday I finally emailed their support and got an answer back that this is pretty simple: just put each command on a new line.

So a potential script could look like this:


on textualcmd(inputData, destinationChannel)

return "
/away Gone for the night
/nick zzZZzz
"
end textualcmd

to set may away mode and also change my nick. It is important to put each of the
command on the very first column of the line, as Textual does not remove leading spaces.

Two helpful links: Textual Command reference, list of IRC commands on Wikipedia.

UPDATE: the "WritingScripts" article meanwhile got updated to better reflect above use case.