How we created IoT system for managing solar energy usage

If you have no idea about the development architecture and mechanical/electrical design behind IoT solutions, they could seem like "having seemingly supernatural qualities or powers". For example, if you show a working IoT system to 18th century people, they'd think it's magic.This article is sort of busting such myth. Or, to put it more technically, about hints for fine-tuning the IoT development for an awesome project in solar energy management area.

Spoiler: no spells, only analytical ideas for developing fast and easy service.

Disclaimer: We offer no one-fits-all solution (it simply doesn't exist), just share the stuff that we find serious. These points may be killer features while on the contrary their neglection may kill it. Pun intended ;)

Introducing IoT system for visualizing and optimizing solar energy consumption

We had the pleasure to work on a project of improving self-generated energy usage. The core lies in visualizing and optimizing the use of photovoltaic (PV) systems for both installer companies and end customers. So, when Solar Manager is installed as a central hub for distributing the produced solar energy between household devices, you use maximum of self-produced energy.

We've already written about business perspective of using solar energy, and now let's dive into tech challenges and hints of the development.

Absolute prerequisite

Before we start, let's get clear. There is one thing that does apply for each and every project, regardless if it's an IoT system or simple landing.

Never start implementation before you have a complete understanding of what you want to achieve (and everyone involved does).

There is no such a thing as evident. No, there are no telepaths, and nobody reads your thoughts. Clarify, discuss, make sure you're on the same page.

Of course, if you don't want to spend twice as much time and money.

The mission

So, what we have as inputs? Flexible connectivity and centralised remote firmware, 5 user roles with different UX and permission sets, converting and analyzing data, huge amount of requests, and, as an icing on the cake, plotting lots of time-value data on charts.

Challenges and specifics of implementing IoT system for energy consumption

When it comes to development, 3 things matter the most: stability, relevance, and speed. Especially when the project requires a monolithic architecture. That's why the overall logic, all user flows, and database design should be as simple and optimized as possible. Let's be honest, in 2019 nobody wants to wait 40 seconds because 'your data is being processed'. So, here's the question/answer list for IoT platforms we refer to:

• Does the architecture fit in the requirements and specifications?
• How will you convert, process, and analyse collected data?
• What are the most optimal relations between parent-child MVC/MVP layers? Can they then be scaled without being interrupted?
• Are there any modules that perform similar/adjoining functions? Can they be united into a single multipurpose module? Will that introduce or reduce complexity?
• Is the communication between stages optimal?
• How to cache data before visualizing it?
• Is the user experience (UX) simple and self-evident for each user role?

Split what needs to be distributed, unite what is similar. Do not confuse this, however, with creating huge classes that (should) solve several tasks. Instead, create small ones for a single task. For complex task, create several classes that each solve a subtask and another one that will use those ones.

• ### Architect IoT solutions smart

Smart here refers to creating fast, scalable, optimized and sustainable project architecture that fits into project requirements. The basic questions that arose at this point:

1. What is the easiest and most direct architecture flow?
2. How to make the user experience for each role the simplest?
3. Can we revise initial architecture for better scalability and performance optimization?

For web apps, this involves the hierarchical model view controller (HMVC) design. For example, to allocate resources better and provide faster response, work through the connections between API, back end, front end, and edge. Configure cloud-local communication, storing and transferring data so that you get right response fast (hello, Captain Obvious!).

• ### Design database with performance in mind

Never underestimate the importance of thinking through the database (DB) design. This also applies to the relations between collections in the database. You have a lot of entities, which interact in various ways, so it is crucial to keep all related logical connections in one place.

So, when you have initial structure at hand, determine:

1. Is this (one-to-one, one-to-many, many-to-many) the optimal relationship type for this collection? Does it solve the task most efficiently?
2. How well does it correspond to the task at specific screen?
3. Does it fit well in the general architecture of the project?

Optimize the logic behind each module and ensure that the general architecture is preserved. Think through the permission set. Then think again and optimize. Remember to set connection types so that for each task you have all needed info in one place and handle requests right.

For example, imagine that your database is a fast food restaurant. You have a lot of clients, you need to serve food fast and precisely what they've ordered. Or else your business is done for. That's exactly the same for DBs. Yes, there are dozens of collections and you need data from 7-8 of them simultaneously. So design the relations so that they're logically connected and return the response quick.

IoT system is like a restaurant. Don't make the customers wait.

Photo by DREW GILLIAM on Unsplash

• ### How to plot [gazillion] of data

Remember the times, when sites loaded for a couple of minutes? We've seen apps where dashboards load even slower. And that's a no-no.

Charts are a new normal. Data visualization is commonly used in SCADA, IoT, and any analytical projects and for a reason. Seeing spikes in energy consumption, market trends, CO2 in the air, and any other stuff you can measure (typically, over time) gives a vivid impression. You see, you understand, you act. With textual presentation of massive data arrays your eye will blur in no time.

So, the challenge lies if fast plotting of charts with huge number of value-time pairs. Before users see the data on the chart, system has to determine all the points for plotting it. We had quite a lot of brainstorming to determine how cache the data so that plotting a chart with 50K+ data-value pairs takes less than 2 seconds.

From scratch to production, and beyond

Apart from web app, we also develop iOS and Android app for on-the-go installation, monitoring, and access.

The important thing to remember when working on IoT systems is maintaining simplicity and scalability. The typical 5-stage IoT architecture should be well-thought to not bring any redundant complexity.

As each product evolves, you have to think through the means of layering new functionality while not interrupting the existing features. And, of course, never forget about unit testing.

Less than a year from launching

Since its launch in October 2018, the Solar Manager is adopted and used by the leading photovoltaic wholesaler and installer companies in Switzerland.

By reviewing and optimizing energy-use habits, the consumers get the best out of their PVs and so they save costs. On a more global scale and according to Deloitte, such a solution makes solar energy competitive with traditional power sources. This, in its turn, makes it appealing for other residential consumers and businesses. And, finally, doesn't it sound as a basis of a smart city we all wait for?

Behind success any IoT project

There is no 'ultimate guide' to creating successful product. First, each project is unique and brings its own challenges. Second, even if you've done a bunch of projects in the area, the tech and the market demands evolve. So the experience you have from 2 or even half a year ago might not be relevant now.

Still, we do not get tired of telling that forewarned is forearmed. So here's a bunch of our experience that may be handy:

• How investigation saves your time and reduces risks
• How to tailor development to market you're reaching
• IoT opportunities for gaming, automotive, retail, agriculture, industry etc.
• What are the top 10 sensors for IoT

You can also check our GitHub for lots of free open-source instruments for IoT development.

Have an idea in mind and want to drill down the tech details? Get in touch with our experts, so we can figure out the best means and tools for the development.