Solar PV Monitoring/Automation Thread

bullit_dodger

I think you guys with the solis inverters have a harder time of it than I do. Reading registers etc, sure it's doable, but it would seem to be a lot harder than parsing or posting a bit of JSON. I'm impressed at the work that your doing to be fair. Not sure I would have the patience for it. That said, I do think I'll have a look at what you posted Jonathan as for those times where the data in the cloud for me are zeros, it might be an "alternative channel" for me to grab the live data. Dunno - if it's a lot of work I might just keep what I have. My biggest issue with all these things is the fecking authentication methods that you have to figure out. Once you get that sorted the rest is easy, but getting that first "success" is the tricky one.

yankinlk, so the idea that I have was to send off a API call to Met Eireann (this part I've done like as follows)

That will return a "Radiation" column for every hour. I then add all those up and come up with a Radiation value for the day. What I've highlighted in red above.

What I'm planning is take the average radiation forecasts over the past 10 days, along with the "actual" generation from the inverter to work out what the expected yield in KwHr is for the day. You have to average the last 10 days, as with the way the sun changes throughout the year the angles that you ha with your panels mean that your production will wax/wane through the year, so you can't just say 1000 radiation = x Kwhr. I can then use the battery API to set the battery level to charge to.

Now I should say, I did the math on this. You might save a tenner a year optimizing the battery charge levels. Right now it's 100% charge at night for me, and in Feb it's likely to be the same, then very quickly for maybe a month in March you might have a couple of marginal days where it's debatable what to charge from night, and then come April, it'll be charge from panels. For the days in march you could adopt a "just set to 50%" and back both horses. So the money you save doing all this optimization work is probably a tenner or so a year.

That's not why I'm doing it of course. For me the value/fun is in the doing and getting it to work.

yankinlk

Thanks @Jonathan and @bullit_dodger I am so impressed with what both of ye have done (or are working towards!) this forum is great. I will liekly go the lazy route this year - ie in march cut back the charge rate manually... its easy for me - no need to climb the attic to set it.

reklamos

https://www.boards.ie/discussion/comment/118460091#Comment_118460091

Playing with registers is only needed initially. Once you figure out what data you need then you can format it in any form you want. In my script after reading all the registers that I want, I create JSON object that ontains all the data and push it to MQTT. Everyone can decide how to present the data be it JSON, CSV etc. At hardware level it is still modbus. Reading cloud API could be sufficient for some but for me the lag, data aggregation and dependency on the cloud was no enough.

Now since you working on pulling data from met. Can you share the info how you do that, I see some urls but have not looked how ti actually works.

bullit_dodger

That one is easy mate. Met Eireann's API takes the Latt/Long as a parameter and returns you some XML. E.g. fire this into your browser (or click the link below)

http://metwdb-openaccess.ichec.ie/metno-wdb2ts/locationforecast?lat=53.27;long=-6.32            

Just change the lat/long values for your location and you should get back a localized forecast for your location. If you can sort out the registers stuff (which I don't care what you say - it's hard), you'll have no issue with the Met Eireann stuff. That negative value for "long" looked wrong to me too, but west longitudes are officially referenced that way.

I've been storing the xmls it returns once an hour every hour since August. Some 3,200 records at the moment. From that I'll be able to average out the radiation levels for specific times of year, see how the forecast for tomorrow have changed over time. My original degree was in mathematics (not computers) so while I don't call it "fun"..... it's kinda interesting to me to understand how it works. But yeah, not for everyone I grant you :-)

SD_DRACULA

In your travels, did you guys manage to update the discharge SOC via the script?

Currently the battery is set to discharge to 30% but starting to see it fill up all the way recently so was wondering how to update it back to 10% via the script.

Any idea which register it is and can the script do that?

Jonathan

https://www.boards.ie/discussion/comment/118461332#Comment_118461332

Yes, you can do that using Modbus. Use function code 3 to read the registers and function code 6 to write.

43010 -> overcharged_soc

43011 -> over_discharge_soc

43018 -> strong_charge_soc

john__long

So glad I found this thread! :D Have been scraping the data every 5 mins of the cloud and pushing to Grafana. Something more real-time, as well as pushing commands back up into the attic, would make my Home Assistant more fun! :D

Anyone know how to update your WiFi dongle, or if you can? Getting this error back:

Cannot read registers 33022 length 19 V5FrameError('V5 frame contains invalid a Modbus RTU frame prefix'

I've setup a RPi with the official touchscreen on the hall table so I can see what's doing on. All powered by Home Assistant running on device, with some help from a small MQTT server in Azure.

Have been experimenting with the domestic usage, i.e. excluding power going to charge the batteries but it's a bit rough. The time lag of 5 mins doesn't help.

reklamos

https://www.boards.ie/discussion/comment/118463738#Comment_118463738

What inverter model do you have? It could be that the registers on your inverter do not match. Try to read single register with sample code. Or loop through them this will give better picture where the registers are.

john__long

https://www.boards.ie/discussion/comment/118463799#Comment_118463799

Not sure of the exact model name. One of these RHI-3.6K-48ES-5G : https://solartricity.ie/static/wp-content/uploads/2020/11/Datasheet_RHI-3-6K-48ES-5G.pdf

SD_DRACULA

https://www.boards.ie/discussion/comment/118463799#Comment_118463799

You want to get yourself one of these for live power consumption https://shelly.cloud/products/shelly-em-smart-home-automation-device/ it hooks right into home assistant very easily.

https://www.boards.ie/discussion/comment/118463035#Comment_118463035

Not getting the right numbers (or I am and they are not converted right?)

    print(modbus.read_holding_registers(register_addr=43010, quantity=1))

    print(modbus.read_holding_registers(register_addr=43011, quantity=1))

    print(modbus.read_holding_registers(register_addr=43018, quantity=1))

And I get back these 3 values:

[100]
[19]
[10]

I expected 29 as that is what I have my min SOC at, battery is in backup mode so it stops draining at 29%

Any idea why I'm not seeing that? Unless min SOC is 10% and 19% is the other charge it reserves for AC backup???

Jonathan

https://www.boards.ie/discussion/comment/118463819#Comment_118463819

Turn on verbose mode and PM or post up the SEND and RECD frames.

In other news, what screen/device is that you're using? Anything more specialised than an old tablet? EDIT: just reread your post. Looks good for RPI. Is that custom mount/frame?

reklamos

https://www.boards.ie/discussion/comment/118463828#Comment_118463828

If you have battery reserve configured then read 43024

SD_DRACULA

Yup that one does come back with 29 which should be correct.

So for updating is it just this:

print(modbus.write_holding_register(register_addr=43024, value=10))

If I want to change that 29 to 10 for example?

john__long

https://www.boards.ie/discussion/comment/118463834#Comment_118463834

This is the mount for it: https://thepihut.com/collections/raspberry-pi-cases/products/raspberry-pi-official-7-touchscreen-case

Will experiment with the modbus script again in a bit. Thanks for the pointer!

SD_DRACULA

Best for home automation control is a 10" tablet and hook it up to the 12v on your alarm panel via a 12v to 5v adapter behind the wall for a seamless integration.

graememk

https://www.boards.ie/discussion/comment/118464138#Comment_118464138

I dont need it, i dont need it, i dont need it... - Its out of stock (the screen)

cromelex

https://www.boards.ie/discussion/comment/118460091#Comment_118460091

That is obviously impressive, and not sure if you've seen this before, but there is an Home Assistant ingreation that already does this. I haven't used it myself yet (no PV yet) but even if you want to do it yourself you might want to have a look and check the logic they are using.

https://www.home-assistant.io/integrations/forecast_solar/

SD_DRACULA

It's hit and miss, I've been using it for a while.

Today it's a total miss, overcast for the whole day in Dublin I'd say and estimates 10kwh 😬

john__long

https://www.boards.ie/discussion/comment/118463834#Comment_118463834

Took me a while to get back to this. Verbose mode enabled.

Totally new to modbus. Going to have to do some reading. Every day a school day! :D

2022-01-15 11:40:53 DEBUG   Scrapping registers 33022 length 19

SENT: a5 17 00 10 45 00 00 2a 46 dc ef 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 04 80 fe 00 13 f9 f7 2f 15

RECD: a5 39 00 10 15 00 38 2a 46 dc ef 02 01 50 a2 fc 02 2d 51 00 00 75 10 e6 5e 01 04 26 00 16 00 01 00 0f 00 0b 00 2e 00 2b 00 00 00 00 0d 39 00 00 00 0e 00 00 00 12 00 01 00 03 00 00 00 0e 00 00 08 d3 0f 88 aa 15

2022-01-15 11:40:53 ERROR   Cannot read registers 33022 length 19 V5FrameError('V5 frame contains invalid a Modbus RTU frame prefix')

john__long

https://www.boards.ie/discussion/comment/118463834#Comment_118463834

My serial begins with 402. Seeing this being referenced for 402 serial WiFi dongles: https://github.com/t3kpunk/Omniksol-PV-Logger

Jonathan

https://www.boards.ie/discussion/comment/118479133#Comment_118479133

Thanks for posting this. This is a bug in the pysolarmanv5 frame validation. At the time I understood that all modbus RTU frames were prefixed by 0x61, but I was doing more digging during the week in the SOLARMAN Smart app, and this is not true. I've pushed a commit to fix this now. I have to further work on the validation before the next release, but the latest code from master should work for you now.

Below is the Modbus Response that was returned from your logger. All looks good. https://rapidscada.net/modbus/ModbusParser.aspx

john__long

https://www.boards.ie/discussion/comment/118479484#Comment_118479484

Thanks for having a look. I’ll circle back to this project in a bit. Would need to refactor some things in Home Assistant to bring this online.

I get so hung up on what to name my MQTT topics! 😅

graememk

I need to stop playing with home assistant and go to bed lol.

reklamos

https://www.boards.ie/discussion/comment/118492549#Comment_118492549

You can add more :)

john__long

https://www.boards.ie/discussion/comment/118479484#Comment_118479484

Thanks for bumping the PySolarman project with the v5 update. Gonna have a play with this again this weekend!

tnegun

Finally got some time to look at this while the inverter was on! Thanks so much for PySolarman I was able to hack together some python quick enough to read the current AC kw output and push this data into my EmonCMS as a new feed. I'll keep it like that while I play with it. I'm polling every 10 seconds and get the odd error returned but am handling that.

The next steps for me are to only poll white the inverter is on and then add the metrics from both my arrays. I notice that the inverter returns 20w once it's powered up and again for while before it powers down but I doubt it's actually generating anything is this a known behaviour?

ECO_Mental

Just got my Shelly EM a few days ago (after waiting 3 weeks to be delivered..) and got it installed into the main fuse board yesterday. To be honest the thing I'm most impressed is I am still alive to tell you guys the tale...🤗 The Shelly EM requires you to hard wire the power directly into one of the trips and then attach your CT clamps all within the main consumer unit. With two live feeds the Solar PV and the main ESB you had to be VERY Careful. I tripped the main switch and the isolator for the PV but still....

Anyway got one clamp on the main ESB incomer and the other CT on heat pump. I already have the Solar PV data from the cloud so I thought there was no point in logging this. I put the CT clamp on the incomer before the main trip switch and before the PV feed so in theory it should read negative values if I am exporting as I had no exposure of this with my basic Solis inverter.

The PV has been a disaster over the last day so I don't I think I exported anything but if you look carefully at around 11:30 am I exporting about 50 watts for about 10 mins...so I think it works.

All my data is being managed through Home Assistant on a PI4 and I'm using InfluxDB and then Grafana. HA picked up the Shelly straight away so I was impressed with that. The graph below shows in green the total house electricity consumption and the yellow is the heat pump and the blue is the Solar PV. Only getting used to Grafana so need to fancy up my graphs a bit.

What shocking is the heat pump is about 50% of my energy consumption atm. and most of that is is heating the water in the 300 litre tank for the 30 minute showers my two kids have at least once a day if not two 😥

tnegun

So my initial script is working well, it needs a serious amount of refactoring and some additional redundancy built-in but it has run unattended for 24 hours. This is today's output solar/orange is the Emon Pis CT clamp(it was reading about 10w in excess so I manually calibrated it) and solis/blue is being polled from the inverter. I was polling at 10 seconds but something funny was going on with missing data so for now it's at 5-second intervals. Probably overkill but it hasn't broken anything yet! The phantom 20w shows as soon as the inverter powers up and before it shutdown but this is present on Solarman cloud too. I'm happy progress so far!

Solarman site for comparision

graememk

https://www.boards.ie/discussion/comment/118533318#Comment_118533318

Ah the missing data!

When you setup your feed, set it to 15 seconds not 10

tnegun

Thanks for that. Mine were set to 10 seconds so I'd kind of stumbled on it with the polling frequency. I still get the very odd missing data but I suspect that's my crude error handling where if I get an error from the pysolarmanv5 function (is the correct term?) I back off and wait 5 seconds before trying again so the interval exceeds 10 seconds. I don't get too many and now that the polling interval aligns with what the feed expects it's 99% good.

Can anyone see a register to pull what kw each string attached to the inverter is generating? I can get the DC V and Amps so may end up calculating it but was wondering if anyone figured out if it was available from the inverter directly.

Jonathan

https://www.boards.ie/discussion/comment/118536924#Comment_118536924

Yeah, the data logging sticks send an occasional unprompted "keep-alive" frame which kinda mucks things up. I have an idea on how to implement error correction to handle them without raising exceptions. Tied up on other work at the moment so on the to-do list for now.

As regards the power of each string, I don't think this exists. Only a DC total power. You'll have to calculate them I think.

graememk

https://www.boards.ie/discussion/comment/118536924#Comment_118536924

Yeah you'll have to calculate it, but it's simple when you have them in emoncms anyway

ECO_Mental

Happy with the Shelly EM and my set up and it records the PV that goes back to grid as a negative value. Looks like I had a bit of a problem but the Solis PV data as it wasn't downloading for the first half of the day. I think the issue was the HACS integration in HA which had a Solis update and I never restarted HA to get it working. Anyway The graphs here really show how much the A2W heat pump is the main driver of energy in my house.

reklamos

https://www.boards.ie/discussion/comment/118536924#Comment_118536924

on Hybrids it is 33057-33058, it shows total DC input power in Watts

garo

I have a Growatt inverter and am interested in pulling the data into a grafana dashboard. The Growatt website used to provide historical data until late last year but they took away that functionality and now I don't have historical records any more. Additionally I also have a Seplos BMS that I would like to monitor/control from my desktop. I am assuming the latter would require NodeRed and a pi or similar.

I could scrape data by polling the growatt website periodically. Or I can change the server address that the RF logging stick talks to and consume the data locally (and optionally forward it to the original server). Any thoughts on what might be the better option? I am loath to add a second point of failure as I have been having some issues with my local network. The server connects off a powerline adapter (I know I know! I have a roll of Cat6 waiting for me to install).

I think a number of people here have a Seplos BMS already. Ideally I would maybe have a Pi that talks to the Seplos on the RS485 port and allows me to measure and control it remotely.

Thoughts/suggestions?

Jonathan

https://www.boards.ie/discussion/comment/118556399#Comment_118556399

Check to see if the Growatt inverter has a Solarman data logging stick. If it does, you could possibly use pysolarmanv5 to poll it remotely without any requirement to mess with existing monitoring or wiring. Check to see if port 8899 is open. That is a good initial indicator.

Don't have a Seplos, so can't help in that regard.

reklamos

https://www.boards.ie/discussion/comment/118556399#Comment_118556399

So my plan for Seplos is to use elfin Modbus RTU to TCP adapters as they are compact, cheap(15Euro) and you can have them in with Ethernet port or WiFi. I have used them in the past and they work very well. RPi is also an option but for me it is too much hassle.

For Growatt, pick what suits you best. If you want upto date and detailed data then pull it from inverter, if that is not important then scraping Growatt website can be sufficient for you.

reklamos

Did some coding today and was able to come with a simple code that allows me to change Solis settings from nodered. The main things I needed to automate was timed charging and also update inverter time, since mine drifts overtime.

Testing looks good so far. Will see how it will work overnight.

garo

https://www.boards.ie/discussion/comment/118556457#Comment_118556457

Growatt uses a proprietary comm device. https://www.alternergy.co.uk/growatt-shine-link-wireless-monitoring-device

https://www.ginverter.com/Monitoring/10-664.html

I have the Shinelink option here so the stick uses RF433 to talk to the box which communicates using the internet. Port 8899 is not open.

garo

https://www.boards.ie/discussion/comment/118556501#Comment_118556501

The elfin Modbus seems like a good idea. I did some preliminary research on the RPi route and you are right it did seem like a lot of hassle.

Let me try scraping the website periodically.

garo

Yaay! Found the API documentation: https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

And some code: https://github.com/indykoning/PyPi_GrowattServer

SD_DRACULA

https://www.boards.ie/discussion/comment/118556530#Comment_118556530

This is interesting alright, is it running via Home Assistant? Assuming it can be used to change the discharge % etc as well?

Maybe share the flow with us 😀

reklamos

https://www.boards.ie/discussion/comment/118558429#Comment_118558429

Here it is https://github.com/NosIreland/solismod it has worked as expected overnight.

You can add registers that you want to modify and create your own naming convention for topics. I use NodeRed for my automation and the example provided is with it, but anything that can publish to MQTT can be used. As long as script can subscribe to a topic on your MQTT server it should work.

Jonathan

https://www.boards.ie/discussion/comment/118558997#Comment_118558997

Great stuff. Glad to see you're using Modbus FC06. 😉

I found a bug last night which means that FC16 will only work for a particular message size. I'll fix that in the next release.

reklamos

https://www.boards.ie/discussion/comment/118559278#Comment_118559278

I cannot think of a scenario where FC16 could be used in automation.

BTW I have submitted issue https://github.com/jmccrohan/pysolarmanv5/issues/2 for writing single holding register also. Have a look when you have time. To get away I just commented 'value = self._format_response(modbus_values, **kwargs)' in 'write_holding_register function'

Jonathan

https://www.boards.ie/discussion/comment/118559926#Comment_118559926

Thanks. Have fixed both that and the FC16 issue now with v2.3.0.

As regards FC16, it is handy to be able to sync the datetime values in one go:

timestamp = list(map(int, time.strftime("%y %m %d %H %M %S").split()))
modbus.write_multiple_holding_registers(register_addr=43000, values=timestamp))

   

ECO_Mental

HA and the Solis Portal integration is being give me trouble over the last week or so, it seems to be getting a daily version updates now from HACS and they cant seem to get it to work consistently. I have not being getting readings for the last few days

To be honest though I am thinking of getting another ShellyEM and to monitor my PV with that. I am impressed with the one I have and it gives you instant live readings and it would integrate nicely with my existing ShellyEM readings. Also the second channel in the shelly I can log my utility room plug circuit which has the washing machine and the dryer and a few other bits and bobs so its worth logging this circuit.

Only issue is possibly waiting three weeks for it to show up

SD_DRACULA

I have been using a node-red scraper that connects to Solis portal and didn't have many issues bar the odd time when the data is not refreshing on their side during maintenance etc which will happen to either solution anyway.

https://flows.nodered.org/flow/1a1e2bac1ecfe8f40527ad930fe788f0

graememk

https://www.boards.ie/discussion/comment/118574603#Comment_118574603

Only drawback is that you'll not see the solar only. Just what comes from the Inverter be it solar or battery.

Have you tried talking to the wifi logger directly with code from Jonathan and reklamos has done a docker container that publishes to mqtt..

tnegun

I think direct interrogation of the Wifi logger is the way to go you can get so much more info than just the current power from the shelly. I'm pushing it into Emoncms then pulling that info into HA. The solis feed is polled directly from the Inverter and each of my strings PV1/PV2 watts is calculated from the voltage and amps for those strings again pulled directly from the inverter. It should be much more reliable and accurate than depending on pulling from the Solarman site.