Hackathon: Node Information Service

At work we occasionally have a Hackathon day where everyone in the lab is able to work on any project they like whether that's just practising some Test Driven Development(TDD) or creating something that could help with your day to day work.

The Idea

So for this Hackathon I got together with Adrian Moldovan and Tom Fletcher to work on a project that would hopefully address a major issue we have at work.

Our problem is that our product has many different platforms and we don't have all the different platforms in each of the teams so we often have to share our hardware. The problem with this is that in juggling the hardware between teams some hardware can often get lost along the way.

The major problems we have are:

• Finding the a specific platform that you need to test.
• Finding hardware on a specific code level.
• Finding hardware that is in a good state for you to borrow.
• Figuring out which hardware isn't being utilised.
• Finding out who you should talk to about borrowing a piece of hardware.

Finished Product

As you can see we have quite a variety of kit on many different levels and this information is invaluable for quickly finding the piece of hardware you require. 

Tech

So let's talk about the tech that has been used for this project. Firstly I've been reading a lot about microservices and was intrigued by this concept of splitting out what would normally be on large application into smaller portions. By no means is this a microservice based architecture for this project as it lacks some of the required infrastructure to be that way.

This project is split into two smaller pieces:

1. A Python REST API which performs the same ssh command to multiple machines at once and combines the result into a single response.
2. A React Web Frontend which displays a table of information for multiple machines.

Python Flask REST API

Multithreading

So the basic idea for the project was to run many of the same ssh command to all of the machines and get a point in time capture of all of the state required to solve the above problems. To achieve this we decided to use threads to be able to kick off multiple ssh sessions at once.

I tend to avoid using multi-threading whenever I can because it massively increases complexity but in this situation it seemed like the correct thing to do.

Stateless-ish

So I really liked the idea of the REST API being a really simple worked that simply ran commands on multiple machines and output the result.

I decided that we should try and keep this fairly stateless and so the only real state that the API has is a simple in memory cache which wouldn't even be necessary if it wasn't for the sheer number of machines that we are trying to perform the commands on.

The caching itself is only held for 15 minutes for each machine to ensure the data isn't too stale.

React Frontend

So this portion of the project is where all the shiny things happen, without this the user would need to be able to easily read and interpret JSON data which is not a skill many people have.

Search and Filter

For ease of use we added a search functionality along with some filter buttons which will allow the user to quickly narrow down the choice that they are after. There is also a refresh functionality which will call to the API and tell it to do a full refresh not including cached data. This is mostly for people who really need the most up to date information.

Infrastructure

Docker

For this project we really wanted to try out using Docker as we wanted to be able to deploy these two applications as simply as possible. After all this was a Hackathon and so it is a perfect opportunity to try and harness new skills.

Both projects were there own separate containers and the frontend simply needed an argument on run which passed in the address for the API.

Lessons Learned

I think the project went really well and everyone we presented it to was really impressed and wanted to see what else we could get out of it.

Architecture

I think the biggest things I learnt were to do with application architecture specifically how this style of application level separation of concerns allows for extendable and reusable functionality. The API itself can be used by any frontend client. If React is no longer the flavour of the month then a new frontend could be made, or if someone wanted this functionality on mobile then they could still simply call the same API.

Containerisation

This whole idea of containerisation I now see if incredibly powerful, it is much easier developing on the same platform that you are deploying to and also this ability to deploy anywhere that has just the one dependency of being able to run Docker is so useful. 

I will definitely be doing a lot more with Docker in future.

Improvements

There are always improvements that can be made, specific ones I would like to look into are:

• Support for the OpenAPI /Swagger API Specification.
• Adding Docker Compose support for deploying both of the Containers together.