Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie
Hi there,
There is an issue with role permissions that is being worked on at the moment.
If you are having trouble with access or permissions on regional forums please post here to get access: https://www.boards.ie/discussion/2058365403/you-do-not-have-permission-for-that#latest

Solar PV Monitoring/Automation Thread

2456744

Comments

  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    You are right got confused here as was trying to apply logic :)



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    If I was to poke all the ports from 1 to x? How high should I go. 50k? 😂

    Will scan the holding registers too see what sticks.



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos




  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    Hmm it doesn't work when I go full ham on it. Maybe some ddos prevention on it.

    But I've got most of my values pinned down now though anyway. I don't think I'm missing anything. (Not that I need it either just fun)


    Oh @tnegun is yours a 4G or 5G inverter?



  • Registered Users, Registered Users 2 Posts: 7,522 ✭✭✭SouthWesterly


    Is it possible to do this without being a programmer?

    Did some coding years ago and absolutely hated it.

    What hardware would I need to buy?



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    Yours is goodwe?

    You could start with just a PC/Laptop with Python installed. But not sure how plug and play it is.

    But yeah it's gonna be quite hands on right now, I can talk to my inverter, and get some information from it, and can figure out what most of the values mean.

    (It's the best way to actually learn to code when you need to do something)



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    It all depends what you want out get out of it. Don't have to be programmer(I am not programmer myself) just need PC/Laptop. Install HASS and play around with it. There are good guides/manuals online with massive community. As with any DIY, we do it because a solution either does not exist or it is much cheaper. The other major benefit, you can do it the way you want it.



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    Since you have the registers now you can create/modify registers list and start sending data to mqtt/hass or if want fancy dashboards prometheus/grafana.



  • Registered Users, Registered Users 2 Posts: 1,755 ✭✭✭tnegun


    Mines a 4G I've been out during daylight hours the last two days so haven't gotten to do much since finding the registers and back to work tomorrow so will be a while before I get back to it :(



  • Advertisement
  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    The irony is that I can already do that as I have an emontx and a CT on the feed from the inverter. Being recorded into emoncms. But having the DC stats will be useful.

    Will be looking into prometheus/grafana.. (need to figure out what they actually are)

    Also need to figure out how to integrate it in my system. Mqtt prob, either triggered in node red or something.



  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    Pushed a new version of pymodbusv5 (v2.1.0) to github there which has much improved error checking. Thanks to those that sent debug logs.


    EDIT: @reklamos Looking at your code, that probably means some breaking changes for the error checking you recently added on your side. Sorry!



  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan




  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    Yep that tallies with my inverter.

    I wonder why they are having the control of the hybrids behind a NDA...

    Isnt there some docs out there regarding control of the hybrids already?



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    Thanks, will check it

    It is difficult to figure out which inverters/wifi sticks these apply. I see hybrid listed but the register do not match what I have at the moment.

    Never mind the page says that different model have different maps. Also looks like RHI series in US have different map to the ones in EU. Not sure I understand the logic why Solis try to over complicating this.

    I do no think the documents that floating around are the official ones.



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    It could be because the us is split phase where we are single or 3(on the continent?)


    I suppose it would be handiest for me to loop through, read the registers and publish to mqtt.



  • Advertisement
  • Registered Users, Registered Users 2 Posts: 1,755 ✭✭✭tnegun


    I haven't looked at the document provided by @Jonathan yet but wanted to share this one that Solis sent me today.




  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos




  • Registered Users, Registered Users 2 Posts: 2,472 ✭✭✭SD_DRACULA


    Great work @Jonathan /@reklamos with the solis stuff.

    Still trying to find some time to see if @Jonathan 's code can be made to run via node-red, the readings from the solis server are unreliable and have skewed the data a few times in HA for me and had to manually edit them.



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    The way I'm thinking of doing it is have the script be triggered by node red, and either output a json object or push it direct to mqtt



  • Registered Users, Registered Users 2 Posts: 2,472 ✭✭✭SD_DRACULA


    I have node-red currently set up to pull all data from solis server and then send to mqtt.

    Had no luck in the past in getting python to run via node-red, do share if you ever get it working 😁



  • Advertisement
  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    I have got it to work before but it was just triggering the script, not running "in" Node Red



  • Registered Users, Registered Users 2 Posts: 2,472 ✭✭✭SD_DRACULA


    Yeah could probably do that but I rather not add one extra device like Pi to just run this, node-red runs on my HA instance so getting to run there would be perfect.

    I know it can be done but a bit beyond my skills.



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    Everyone will have different setup but here is how I have it at the moment

    In my setup everything runs in docker(HASS, mqtt, nodered etc). I have created docker image for the script also and uploaded to docker hub. The instructions how to run it is in github.

    The script allows to configure mqtt server ans sends metrics to is as single json object(I had each metric separately but it as too much for mqtt to handle in one go. Instead to single IO it had to take over a hundred at once.

    Mosquito MQTT and nodered are both talking to Hass. I only pull metrics that I want from MQTT to Hass by defining sensor in configuration.yml file

    - platform: mqtt
      state_topic: "solis/METRICS"
      name: "Total house load"
      unique_id: "sensor.total_house_load"
      unit_of_measurement: kWh
      value_template: "{{ value_json.total_house_load_2 }}"
      state_class: total_increasing
      device_class: energy
    

    Once Hass restarts it will create entity 'sensor.total_house_load' and this is where it will pull metric in Hass. If nodered is integrated into Hass then you will also see the same in nodered.

    You can pull from MQTT directly to nodered. Just use 'mqtt in' node and subscribe to the relevant topic.

    As mentioned I only pull basic stuff to Hass as there is no need for all the metric I just want to be able to use the Energy feature and also this on dashboard as it give quick representation on what is happening.




  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    @Jonathan ,

    as I'm coding at night i cant test, but in the bit of code posted earlier to scan the ports, it pulls out

    !!!!!!!!!!!FOUND!!!!!!!!!!! 2999 [172]

    When i call this:

    modbus.read_input_registers(register_addr=3005, quantity=1)

    Does it return a value in square brackets?



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    It is a list so it comes in brackets, if you to change quantity to 2 it would be [123, 456]

    Edit: the output type is a list.



  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    If you need to get a single value rather than a list, use read_input_register_formatted() and read_holding_register_formatted(). These functions will do any combining of adjacent registers for you, as well as signing, scaling, bitmasking/shifting etc.

    @reklamos Just added an updated version of the register scanner to the repository. Made a few tweaks to formatting and to catch KeyboardInterrupt. Previous version was a bit awkward to kill. Had to resort to kill -9. 😂



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    ahhh.


    ok do i have to worry about the last argument for now?

    read_input_register_formatted(self, register_addr, quantity, **kwargs):

    I'm only pulling out 9 values, scattered about the list. I don't think i need to pull everything..



  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    No. The last argument (**kwargs) is a special parameter in Python to allow you to pass keyword arguments (basically optional arguments). See last four example requests examples/client_example.py to see how it is used.

    Post edited by Jonathan on


  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    Thanks for your help, Havent really used python much. most of my experience is in C and C# (and node red javascript!) I find python so "wordy" (and no curley brackets!)

    Have something basic that I can trigger from node red and output some Json back out. As I can already monitor the CT for solar, I can use that to trigger, otherwise id have to either a) ping the logger or have some code to activate at daytime.

    @SD_DRACULA Im just using a execute node to trigger the nodered script, so I'm not a huge amount of help for you there unfortunately.

    Ironically I had moved my node red flows to a plugin/jail on my truenas box, so I havent even attempted installing python on that yet. Running it on the Pi that is hosting my emoncms.





  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    So my code runs fine on my pc, Once I move it to the pi, it fails.

    What am i missing? a dependancy?

    Traceback (most recent call last):

     File "main.py", line 95, in <module>

      main()

     File "main.py", line 59, in main

      thislist = modbus.read_input_registers(register_addr=3005, quantity=38)

     File "/home/pi/.local/lib/python3.7/site-packages/pysolarmanv5/pysolarmanv5.py", line 206, in read_input_registers

      modbus_values = self._get_modbus_response(mb_request_frame)

     File "/home/pi/.local/lib/python3.7/site-packages/pysolarmanv5/pysolarmanv5.py", line 157, in _get_modbus_response

      mb_response_frame = self._send_receive_modbus_frame(mb_request_frame)

     File "/home/pi/.local/lib/python3.7/site-packages/pysolarmanv5/pysolarmanv5.py", line 152, in _send_receive_modbus_frame

      mb_response_frame = self._v5_frame_decoder(v5_response_frame)

     File "/home/pi/.local/lib/python3.7/site-packages/pysolarmanv5/pysolarmanv5.py", line 127, in _v5_frame_decoder

      raise V5FrameError("V5 frame contains invalid a Modbus RTU frame prefix")

    pysolarmanv5.pysolarmanv5.V5FrameError: V5 frame contains invalid a Modbus RTU frame prefix



  • Advertisement
  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    Doesn't appear that way from the traceback. Could just be a transient thing if the data logger was accessing data for Solis Cloud at the time.

    If it always fails from the Pi, turn on verbose and post the SEND and RECD V5 frames.


    EDIT: Could also be a different Python version, but breaking changes between minor versions are rare enough.



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    ah, on pc its 3.9.9. pi has 3.7.3. Will investigate more.



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    While I'm poking around looking how to update python.. and not break other things!

    i tuned on verbose logging, It doesnt even get the the SEND and RECD frames. but a new error has flagged up.

    TypeError: hexlify() takes exactly one argument (2 given)



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    so in 3.7 hexify() is binascii.hexlify(data)

    but in 3.9 (and 3.8) its binascii.hexlify(data[, sep[, bytes_per_sep=1]])

    @Jonathan FYI



  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    Thanks. I will mark as Python 3.8+ for now. If I get a chance I'll see can I backport support for earlier 3.x versions.



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    Yeah currently having a mare of a time trying to get it to work in 3.9.9 on the pi.

    I have 3.9.9 installed but it's not installing your code. I think it's something on my end tho. You'd think they'd have the apt version of python a bit newer!



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    With a lot of faff (I think its because something was screwy.. and python not compiling correctly or something) I have python 3.10.1 installed! and it finally will install your library.

    Testing tomorrow and logging... hopefully



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    Huzzah, Pulling data in now, Will be interesting to see the difference in performance between my two aspects


    Post edited by graememk on


  • Registered Users, Registered Users 2 Posts: 2,596 ✭✭✭bullit_dodger


    I got my own scraping working on Friday. Problem for me is that they aren't like-for-like sizing.

    I guess I could "normalize" as in production per 1Kwp. Though if anyone has any ideas.....shoot.



  • Advertisement
  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    Nice to see both of you got it running.



  • Registered Users, Registered Users 2 Posts: 2,596 ✭✭✭bullit_dodger


    You got 2.4Kw today?

    A meager 0.768Kw for me off 5.3Kwp (but as mentioned split East/West). What's your layout again graememk?

    Edit: 8.6Kwp (NE/SW) - I forgot that it's all in the spreadsheet now. Doh!



  • Moderators, Home & Garden Moderators Posts: 6,539 Mod ✭✭✭✭graememk


    briefly! then it got dark.

    8.6kwp, NE/SW split 50:50, low slope.



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    I had 2.4kW with 4.5kW facing south today. But yesterday was able to hit 2.5kW and generate 7.2kWh. January looking pretty goof so far.




  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    What registers are you using for your battery usage? 33133/33134 (inverter figures) or 33141/33142 (BMS figures)? Seems to be a slight lag with the BMS figures if you are pulling them from inverter. Might not be an issue for you if you are pulling directly from the Pylons?



  • Registered Users, Registered Users 2 Posts: 793 ✭✭✭reklamos


    For this specific graph I do not use any of these. I create my own modified metric(battery_power_modified) from 33135(battery_current_direction) and 33150(battery_power_2). Using this I do not see lag in the graph.

    33133/33134 is not as precises as 33141/33142 and in my case is 0.4v higher. I have compared the values with the one on pylon itself and 33141/33142 is accurate and it should be as it is provided by BMS. There could be lag as we do not know how often inverter is polling from BMS where as 33133/33134 is read by inverter itself on battery ports.



  • Advertisement
  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    Thanks. Yes, that is what I noted too. A bit of a lag on 33141/33142, but your approach of using a combination of 33135 and 33149 is good, and probably enough detail for my needs.



  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    @bullit_dodger Re your post here (I've no idea how to cross-quote on Vanilla): https://www.boards.ie/discussion/comment/118457512/#Comment_118457512

    Did you see this? https://community.home-assistant.io/t/givenergy-battery-inverter-home-power-things/266291/17

    The code isn't published anywhere at the moment, but you can extract the code from the docker image.

    user@host:~$ docker create --name=givtcp britkat/giv_tcp-ma:refactor

    user@host:~$ docker export givtcp > image-fs.tar

    GivEnergy inverters also seem to use High Flying based WiFi/Serial SOCs, but use Modbus TCP rather than encapsulated RTU. Code is in Python and looks to be a decent state.



  • Registered Users, Registered Users 2 Posts: 2,596 ✭✭✭bullit_dodger


    Ahh nice one mate. The way that I've been doing is via the REST API and I grab the data via a bit of powershell.

    # Set system variables
    $baseUri = "https://www.givenergy.cloud/GivManage/api/"
    $json_extration_folder = "C:\Tools\GivEnergy\Cloud_Extract\"
    $filename_Datetime = get-date -format "yyyyMMdd_HHmm"
    $Inverter_date = get-date -format "yyyy-MM-dd"
    
    #---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    # Login to GivEnergy Cloud and create a session. 
    #---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    $loginUri = $baseUri + "login"
    $authMode = "Windows"
    $tokenRequestBody =
    @{
      "account" = "xxxxxx"
      "password" = "xxxxxx"
    }
    $output_filename = $json_extration_folder + "Login_response_"+ $filename_Datetime + ".json"
    $login_response = Invoke-RestMethod -Method POST -Uri $loginUri -Body $tokenRequestBody -SessionVariable "MySession" 
    
    #---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    # Download the Inverter API information
    #---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    
    # Get the getInverterRuntime
    $loginUri = $baseUri + "inverter/getInverterRuntime"
    $tokenRequestBody =
    @{
      "serialNum" = "xxxxxx"
    }
    $output_filename = $json_extration_folder + "InverterRuntime_"+ $filename_Datetime + ".json"
    $getInverterRuntime = Invoke-RestMethod -Method POST -Uri $loginUri -Body $tokenRequestBody -Websession $MySession -Outfile $output_filename
    
    

    and then it returns a JSON file, which I then dump into a FoxPro database (old skool me) and I can slice/dice it at will. I don't have a "proper" JSON deserializer, so I had to write my own.

    With the GET api's I can modify the charge setting, timings to charge/discharge etc. although I'm not doing any of that at the moment. One of my pet projects which this is leading me to is interrogate the Met Eireann API and then figure out a "expected" generation for the next day and figure out the battery charge level.

    In theory like a few others I could do a few things with smart plugs - but idea in the infancy with me at the mo.



  • Registered Users, Registered Users 2 Posts: 2,752 ✭✭✭yankinlk


    I was gonna post up had anyone worked out how to do this yet... come late March or April time I can see needing to Adjust night charging power so I don't end up over filling battery back to grid.



  • Moderators, Education Moderators, Home & Garden Moderators Posts: 8,253 Mod ✭✭✭✭Jonathan


    You've a Solis Hybrid, right? In that case, you can do this already using pysolarmanv5. You can get details of the Holding registers from Solis Support under NDA, or someone from TCD has translated them and posted them here. If you're handy with decompiling APKs, there is also a complete listing embedded in the Solis Cloud APK.

    I am not aware of a way to target SOC on Solis Hybrids, but you can emulate it by reading current SOC, and then setting charge rate and charge duration accordingly.



  • Advertisement
Advertisement