Back to Smartcitizen.me

Building the firmware

Firmware
#61

Ok, I have done a little reading and discovered:

The (somewhat oddly named) text section (segment) usually contains the compiled binary code, as you suspect. An executable often contains other types of data besides the code, like initialized variable values, resources, debug information, relocation data, and so on, which are placed in sections with other names.Dec 13, 2014

So this means that my discovery that the compiler thinks that my program code is too large to fit in memory not just the ascii text found in sprintf statements (although that’s one thing that’s contributing of course).

In a way, that makes my job to fix this issue a little easier. In fact the amount of reduction I might gain by simply removing the debug is quite small; whereas I am seeking a reduction of the order 30 kbytes I think.
So I am looking for BIG things to remove not little things.

My mind turns to the fact that the current code base is designed to support a myriad of sensors and groups of sensors dating back to the beginning of the Smart Citizen project. But many of the ‘legacy’ sensors are no longer used; and many also are only used when the user opts to attach things like GPS, OLED etc
The libraries for these devices are quite sizeable.

This issue is akin to what we see happening with Microsoft Windows. It has grown and grown over the years always betting bigger never smaller; and every of user pays for it in terms of needing to have pc’s with ever increasing resource requirements.
But Smart Citizen Data board resources are strictly limited to the amount of Flash available on the SAMD21G MCU. Sooner or later either we need a different MCU with more resource; OR we need to put the firmware on a diet and send it to the gym.

In fact there is a case for upgrading the Data Board and Urban boards firstly to get more resources and secondly to replace sensors that are proving troublesome. But that is a digression…

So I am thinking that it may be possible to fabricate a mechanism to optionally include/exclude support for certain devices from a final object code build.
It would mean making a different “build” for each combination of supported devices.

The broad structure of the proposed mechanism:

A header file “build.h” might contains a number of #define macro variables; and it is included in all the other header files so the macro switches are visible in all parts of the code.
A given switch would be used to “wrap” sections of code in a “IFDEF” type arrangement. It would need to be carefully crafted not to create broken dependencies all over the place.

Looking at the code base here (below) are my targets for running an experiment to see how viable this proposed mechanism might be.

Targets for this treatment

  1. SCK Gases pro board ( this early version of gas sensor board is not often found)
  2. Battery ( might be difficult as it’s intertwined in a lot of places)
  3. GPS ( there are 3 different devices supported, each with their own library)
  4. OLED (the ug8 library is HUUGE)
  5. Atlas
  6. Aux (all of it, not used in SCK 2.1 systems; only in SCS)
  7. ….

So this proposal has ramifications of course. It would become necessary to distribute different versions of compiled firmware files. It would also make testing of the firmware into a nightmare.

Finally I have to think whether my own code changes have caused this.
I have added blocks of code for:
(a) support for SCD41. It’s a clone of SCD30 with slight alterations.
(b) support for I2C mux. It became necessary due to the attachment of 10 devices to the Aux bus; with disparate treatment of I2C pull-ups. It’s a necessity if you want to use sensors from different vendors instead of designing your own boards to match the system.
(c) support for a second Pm board. It’s a clone of the code for PM board 1 and “slim” because the attached sensors are fairly simple. It is needed because SAMD21G has only 6 UART devices available and PMB#1 has used them up for PM sensors …

All together I’m thinking my code additions couldn’t amount to 30 kbytes worth off code; ( Surely not ?!) but I can verify that by running the “Inspect” command against the current 0.9.8 baseline code from GitHub and comparing the memory map for the two versions. Something I will do a bit later.

For now I will prune off some excessive debug statements and see if the firmware can become passably stable.

#62

Ok

I did some googling on the subject of SAMD51 And found that Adafruit have a couple of Arduino compatible boards with SAMD51 on board. They also have an ESP wifi breakout that can be added. The cost for a ‘Grand Central’ unit plus ESP breakout about USD $50
Adafruit designed this board with CircuitPython in mind; it’s something outside my present state of knowledge, but it’s enough to know they have produced the necessary library etc support to create firmware using Arduino CC; which I assume can also support Platform IO.

The extra Flash programming space should ensure that a debugger version of code could be built to run with the extra resource requirements for debugging symbols etc pushing up the size of the firmware memory footprint.
I do not know if they included a 10 pin connector for ICE debugger or not. I’ll check that.

But someone would need to design a Arduino /seed style {hat/shield} to break out the the many GPIO pins and provide:
(a) 16 pin connection for an Urban board
(b) a bunch of Grove or Kwiic sockets to connect other sensors. This could also include 8 pin connectors for the PM sensors.

It would also be necessary to check out the possibility of this board directly powering sensor peripheral devices; is there enough ? If not then an external 3.3 regulated supply might be needed; plus the 5 v boost supply needed for (say) pm sensors.

Using it would likely negate the need for PM Board as the MCU has enough pins and ram/flash resource to handle what the PM board supplies. It has on board 8 GB Flash that would replace the functionality of plug in Flash (SD card). But it also has the SD Card …
Thus although the Samd51 will use a little more power than Samd21 the overall consumption might not be much greater. This needs checking out I think. Microchip say that SAMD51 has ‘industry leading’ low power consumption quoted as a ratio of clock cycle. BUT the SAMD51 has a much higher clock speed; so I guess it uses more power.

So, it becomes a completely different type of heart for Smart Citizen; perhaps it is useful perhaps not. I would need to collaborate with you on deciding this (for my project). Some things are too big just for one person…

I will maybe turn this into my next project AFTER the current SCS V3 wind/rain project is up and running. Would it have your (Fablabbcn) support ?

Anyway I ought not allow this to distract me from current activities;
I now have PM board #2 functioning via I2C and can reconnect it to the Aux bus; then test its integration and also clean up the data MCU firmware to use less flash as discussed in previous forum message.

#63

At this stage I am finalising the firmware that goes into the second PM Board; its sole purpose is to make readings from the two sensors: Calypso ULP Anemometer and Radeon RG15 Rain Gauge.

I had some trouble getting the second PM Board to work on the I2C bus as Slave; which problem is now fixed.

But Now I want to make certain that the readings flow through to the Master device without intervention of the MUX; and without the ‘noise’ introduced by Data board which has a lot of work to do.

Test Setup:
a) A seeduino V4 Arduino clone. It has firmware that polls the PMB#2 via I2C and runs through a series of requests every few seconds. This is designed to emulate what normally happens in the SCS Data Board firmware. It just cycles through the 6 different readings and then rests for a few seconds.
b) The PMB#2: Its firmware has essentially two threads.
Thread #1 runs on the ‘loop()’ function; every 5 seconds it requests the Anemometer and Rain Gauge to take a set of readings. It stores the result in global variables.
Thread #2 is the two interrupt driven I2C functions: receiveEvent and Request Event. These run one after the other as determined by I2C packets received from the master. If the Master sends a Command; it is received and decoded by receiveEvent. It Master then makes a I2C ‘Request’ for a reading: (making an opportunity for the slave to send data) requestEvent then fires; and typically this function sends either an Ack or a Reading depending on which Command was sent on receiveEvent.

Connected to PMB#2 is an Atmel ICE; which is used to install firmware into PMB#2

Each of the two boards is connected using USB Cables to the Macbook Pro; and I can use Serial Monitor (Arduino CC and Cool Term) to monitor debug output from each of (A) and (B) simultaneously. the rows of debug are time-stamped to microsecond precision. So I can align both ends of a transmission.

Findings:
(a) The Rain Gauge in particular seems to get spurious data on its Uart Comms line. I previously observed this occurring when I had the Oscilloscope connected. I thought it might go away when the Serial wires were terminated at both ends; but this proved not to be the case. I have a new/revised parsing algorithm that helps to deal with some of these issues.
(b) Some Rain Gauge readings seem to get corrupted when sent from PMB#2 to Master device.
a 4 byte Float containing the byte equivalent of 0.00 at the sender is received as 0,255,255,255 at the Master. It is NOT a float and gets interpreted as ‘Not a Number (nan)’.
This appears to happen on random commands; but affecting some more than others.

I set up an additional reSend of commands that failed in this way; and found that most commands received the correct data when resent, but some did not.

Not satisfied by this; I then installed a 'I2CDriver; device interposed between Slave device and Master device. I2C Driver Info
I then found that the above random corruption of readings received by the master no longer occurred. I conclude that what is happening is that the I2C signal between Master and Slave (Slave sending) is being misinterpreted; and somehow ‘fixed up’ by the I2C driver. Unfortunately the I2C driver circuitry is not transparent. Whilst the files are all ‘open source’ and found on Github; I cannot open the Circuit SCH file. (It needs to be PDF to suit me).

Anyway; its obviously impractical to have this I2CDriver form a permanent part of my SCS V3 project; but I might hope that some form of signal fixing might occur due to the I2C Mux unit. If not then I will just have to live with the occasional data corruption issue. SmartCitizen core firmware already has a ‘retry’ mechanism inbuilt to SCKBase; so all my code has to do is check and detect data received and return false if the reading is not good. SCS will have to just keep trying.

#64

OK; having made the PM2 board and attached sensors work on I2C; recognised by the Data Board firmware is another milestone reached.

I moved on to try to stabilise the firmware.

I found that after loading new firmware (no config) that it would enable all of the attached sensors; even those attached to the I2C Mux. But when I executed the ‘config’ command with Wifi Parameters in order to bring the device on line; the system would stop at the instant that it was attempting to ‘getReading’ from the ‘Noise’ sensor. There were no error messages/diagnostics to aid me. I added some; but this just seemed to make the problem worse. When I added more lines of debug to try to isolate where the freezing was occurring; there were less lines of output from the target function.

I conjectured that this might be because of a shortage of resources.
So, I set about doing as I mentioned earlier; that is; disabling sections of code associated with sensors not presently fitted to this system.
This includes ‘Gases Board’ and a few others.
I tried this is stages; each time reducing the ‘flash’ memory footprint by a lot; and ‘RAM’ usage by a little.
I reduced RAM usage from 24868 bytes to 24116 bytes (of 32768 Bytes); and Flash usage from 244244 Bytes to 220700 Bytes (of 262144 Bytes) - from 94% usage to 84%.

This did not seem to be sufficient to fix the symptoms; however I also made some minor change to the code in ‘Noise:getReading’ to correct some anomalies that I observed:

The following illustrates:

	if (rmsSum > 0) {
		rmsOut = sqrt(rmsSum);
	} else {
		rmsOut = 0;
	}

(I added the ‘else’ clause on the assumption that if the sum happened to be negative; the code might barf when trying to compute the square root of a negative number). the rmsSum might be negative if the registers in the device contain negative numbers at the time of initialisation (This would be the first time that the Noise device is being read). In fact another possible cause is initialisation/population of an array of readings that are averaged and used to compute the Noise level. If the system were short of resources then attempt to write to the array might cause a memory ‘fault’ which could simply halt the processor. ← that is a theoretical possibility:: unproven
Without a debugger and a 'debug version of code containing debug symbols; it is impossible for me to find out the exact cause. Which is why I felt the need to reduce the size of the firmware code.

This version of firmware ‘worked’ : eg it got past the point where the system was freezing before; and started to cycle over the different sensor readings. But there was a lot of debug that was clouding the issue. Text spewing out of the monitor screen: cannot tell really what is going on.

So, I systematically went through and removed a lot of the debug. This cured the problem, and now I find that the I2C mux board is no longer being discovered on the Aux Bus. snafu.

This illustrates when I have mentioned earlier; Addition of debug can lead to instability; but removal of debug code also destabilises causes issues; I am forced to reinsert (some of ) the debug code to try to identify and fix whatever issue.
and That is where I am at right at this moment…

#65

Hi @bryn.parrot,

I think we need to have an easier way to go through open issues. Right now I am not sure if there are still problems with the size of the code. What I would suggest is that a tool like github could give us categories in the issues and ways to review and close them. If you are posting the code there, we can discuss some issues on your fork. It’s really not the case that we don’t want to support, but is more that we don’t fully know how…

#66

On another topic, the data part for the spec sensors will be added in the scdata repository. I have just added the issue to discuss there: Implement spec sensors · Issue #11 · fablabbcn/smartcitizen-data · GitHub

#67

I confess to not being completely familiar with regular operation of Github and not used to working with it actively and professionally.

What I am doing is keeping open issues in my head; and sometimes work on several threads at the same time.’

If thats a nod towards working more methodically; you are right.
However I find that sections of the code interact with one another.

Late today I thought I stabilised PM2 board; but then found I2C mux channel switching has started playing up again; witout which nothing works; so I am forced to divert attention to that.

It is a complex system. I am plugging away; hoping to see some light soon.

#68

On another topic, the data part for the spec sensors will be added in the scdata repository. I have just added the issue to discuss there: Implement spec sensors · Issue #11 · fablabbcn/smartcitizen-data · GitHub

OK, Thanks, noted. Please let me know about progress.

#69

OK; at this point: I believe that I have the PM2 Board communication working as well as it can.

My main next task to to try to find out why the MUX appears to lock up.

Symptoms observed via the UI:
When new firmware is installed; the device restarts automatically or maybe it needs reset button to help. When it does so; depending on previous conditions; the system may find the mux during start up; and build the address::channel map. If not; then I have the need to power cycle the kit and reset the hardware.

So, having installed new firmware and restarted hardware; the device has no network config. When starts up; it cycles through all of the possible attached devices and attempts to start them up. Those that start are marked as enabled; those that do not are not serviced during the reading cycle.

  1. Phase 1: startup after reboot/power cycle
  • I find that the Mux finds 12 devices out on the Aux bus working through the mux. At the end of this first stage in the process; the device stops and awaits further input.
  • I am able to enter manual commands.
  • I can command a ‘read’ of a sensor which delivers just one reading. Or use the ‘monitor’ command that delivers a continuous series of read commands. This bit works well.

If I then enter ‘config’ command with parameters for wifi and a token; the device then starts second phase.

  1. Phase 2: After Config entered: readings commence

  • The device opens up ESP network connection and begins to take readings from the sensors. The readings are taken in the same order as the startup phase.

  • Part the way through this second phase; I find that readings returned are null or empty. (Key Finding)

  • Evidently; the firmware is unable to command the MUX to oen the specific channel needed to communicate with a device.

  • I have added debug that checks every channel for connectivity. It is similar to the code that runs during Phase 1 to discover connected devices.

  • I added this debug code to the ‘getReading’ or ‘update’ functions for each devce, starting from the bottom up; eg. last ones first.

  • When I reached NEOM8U GPS I found a possible target: Evidently the channel leading to CPS was open; but at the end of ‘getReading’ the Mux could not be commanded to deliver the debug test of each connected device.

So, I have examined the code for UBlox GNSS and I think I have found a smoking gun.
I see within that code; there are I2C commands like this:

//Send a restart command. Do not release bus.
 if (_i2cPort->endTransmission(false) != 0)

Well I have to say this looks very suspicious to me. Whilst my inspection of the code also finds this:

//All done transmitting bytes. Release bus.
  if (_i2cPort->endTransmission() != 0)

I conjecture that maybe it is possible that the authors of that code might have missed an exit condition; and the I2C bus remains ‘locked’ when the ‘getReading’ function is finished.

So, my next step will be to look more closely at this; Whilst it is not prudent for me to amend the published ublox library; It might be possible to add a line of code to the Data Board’s NEOM8U class getReading function like this:

auxWire.endTransmission();

"just in case" my conjecture is true. If its not true; adding this command should have no effect…

Addition:
I thought perhaps all of my editing of the sckaux.cpp file concerned with adding new sensors and support for the MUX I might have inadvertently removed a line of code similar to the one above.
Therefore I ran a comparison check (of the distribution version of firmware cw my WIP version) using the ‘BeyondCompare’ tool (which I highly recommend); and confirmed that had not, in fact occurred.

NOTE: Later in this development process; I shall be able to use that same tool to revert many of my changes (consisting of debug code mostly); leaving only the essential elements pertaining to the changes I need to make for this project. It is a very handy tool to have in my toolbox.

#70

OK; follow up on the previous post:

I added the code as aforementioned to the end of each function within the NEOM8U class.

I installed in, and found: It made no difference at all.

So, (after cursing) I moved on to the next stage of diagnosis.

I have a routine that was created yesterday. I mentioned it. It goes through each entry in the Mux channel map that is created during startup; closes all channels and then opens each one and tests to see if it can talk to the previously found address.

I had added this routine to the i2c command. Yesterday I also added it to the end of 5 of the ‘getReading’ functions found in classes within sckaux.

So now I removed it from those places, and instead placed it at the top of the auxboards::getReading controller function.

I then compiled and installed this firmware. I found that it in fact appears to have ‘cured’ the problem. That is; the Phase 2 process reaches the end (with a compete set of reading values) and publishes the data.

That makes this another milestone; I think.

The reasons WHY this fixed the problem are conjectural; but I think it is because adding it to the to of the AuxBoards::getReading function means it runs for every sensor; not just the NEOM8U; So, the conclusion to be drawn is that one of the other sensors is holding up the bus; or maybe I really can have only one channel open at a time.

I will leave this alone now for awhile; but before final version it will need to be cleaned up to just the essential commands needed to keep the I2X Aux Bus (Mux) sane.

So now; as is normal: other problems are in evidence:

  1. SCD41 CO2 reading is always zero (whilst it is delivering Temp and Hum readings OK). I have to investigate further because I do not think the sensor itself is faulty; so it must be the way I am driving it.
  2. The Noise sensor again rears its head: On second time it is read; the code appears to hang.
    I know this sensor has been problematic before; because there is evidence that loop timeouts etc have been added to try to stop it from hanging.
    I saw this previously; I thought I fixed it; but no. I also had hoped this problem would be helped by the reduction in size of code footprint; but again no.
    I will revert the code to distribution version; then investigate further.

before I do anything else; I will publish this code into my bryn51 github repo so you can look at what I have done to date.

#71

@oscgonfer @pral2a @victor
2nd Issue first:

I took a look at SckNoise::getReading.

I asked myself what could be going wrong.
It definitely takes the first reading requested; but 2nd call it hangs.

I did two things:

  1. I moved initialisation of ‘source’ array (2 KBytes in size) into the .h file; hoping it would show if there was a memory issue or not (Yes if it fixed the problem). It is only one of several largeish arrays used: I assume that as function variables they are instantiated on the heap. There’s not much spare memory on this system.

  2. I reinstated some debug.

Here is one piece of it:

        base->sckOut("sckNoise:getReading",PRIO_MED,true);

	if (!I2S.begin(I2S_PHILIPS_MODE, sampleRate, 32)) return false;

	// Wait 263000 I2s cycles or 85 ms at 441000 hz
	base->sckOut("sckNoise:getReading started priming of I2S port",PRIO_LOW,true);

I compiled the firmware and installed it.
One second set of readings; the program hangs once again.
The above 1st line of debug appears in the output; the second does not.

I conclude that there is some issue with the I2S ‘begin’ function that allows it to hang.
Unfortunately, the I2S library is part of Arduino tool chain library; not easy to get into it and certainly not possible to make any kind of change to it.

I will do some reading about this; see if there is some discussion out there on the 'net.

#72

OK;

Concerning the issue with the Noise sensor.

This looks like an issue with the I2S library found in the arduino tool chain. It is not something I am going to be able to fix by myself. Looks like @victor area of expertise to me. It seems likely that in fact it has already been fixed; and if we could use a more recent version of the tool chain then the problem might go away. As to how this issue has arisen when I have made all of these other changes to the data board firmware; I cannot say; appears unrelated.
In my version of firmware; I will back out of the changes I made in SckUrban; and retest. If problem remains; then I will simply disable the Noise sensor; it is distracting from my main game. It is stopping me from stabilising the firmware for the changes I have made.

Secondarily what work then remains:

1/. Reset of the I2C Mux at the top of SckAux::getReading - I need to remove unnecessary code; trim down to just the code that is needed; and also ensure such code only operates when a Mux is installed.

2/. SCD4x CO2: I need to figure out what I have done wrong causing the CO2 sensor not to deliver any readings. I am looking at the ‘Atmospheric Pressure’ compensation as a possible cause; its easy to remove. I can play with it for an hour or so.

==================
When this stage has been completed; itis time for a full test of the system; first goal will be one hour of continuous running; 2nd goal will be 24 hours. This should shake out any remaining code stability issues.

3/. Second OLED: The plan for 2nd OLED is to bring it online; and the simplest pathway is to make a slight change to the OLED class allowing I2C address to be specified at the (begin) stage. So then I can instantiate two OLED Objects. When @victor originally created the OLED code; there was not even the possibility of having a 2nd OLED in the system: I2C address could not be set. Now it can;
Then create code that will allow the existing display of readings to be shared between the two displays; in the case where there are two OLEDs.
Instantiation of two OLED objects will establish that the 2nd OLED can work; and there are enough resources for it.
We can make it also possible to have just one OLED but on the 2nd alternate address. Useful in the case where there is an address conflict.
Then look at what can easily be done to support 2nd OLED for other purposes. Such as debug output. [ 1st OLED displays readings; 2nd LED displays debug ] {maybe}. Just for now its enough to do the first stage of it, look at enhancements later.

3a/. Some time ago I was gifted an alternative eVOC/ sensor; (alternative to CCS811) Maybe I could write some integration code for it and make it possible to deploy this in some other kits. Full Deployment in my instance of SCSV3 appears not possible due to mounting issues: It requires exposure to outside air. I would have to modify the enclosure to use it. It requires an aperture of about 1cm square; and it has no mounting holes.

4/. It’s almost time I got the system working over 4G Radio. Until now it has been connecting through our household WiFi. The reason for this is that I found that both of the two WiFi->4G gateway devices that I have incur internet connectivity issues; requiring further chasing. Maybe needing technical support to help me with correct settings. Initially I will go back to factory default settings; see if it can work. Luckily the current firmware allows establishment of WiFi settings on the UI using the config command; so its a quick and simple thing to see if it can work.

5/. Then there is the Solar Panel and Big Battery. To date the solar panel has remained in the box it came in. Almost time to get it out of the box; connect to the battery; confirm it can charge the battery; and how long it takes.

So, in summary; there is a little way yet to go on this project; great strides have been made; but it has taken a lot longer than originally envisaged. There have been blockage issues crop up all the way along the path. (e.g. the I2c issue leading to deployment of the Mux). Would you believe I originally estimated I could get the firmware going inside a month !? So far more like 4 months and counting. But at least there is now some light in the tunnel.

6/. The final hurdle when all that is done; will be to try deployment of the SCSV3 in some situation; see how it works when it is left alone (unaccompanied) to hum and click to itself out in he middle of a National Park. First step will be to enlist cooperation from the park authorities allowing it to happen. I imagine a bureaucratic miasma; but you never know your luck in the big city; could be easier than I imagine. They might even get enthusiastic when they learn what I am trying to do.

7/. When I have proven to my own satisfaction that the system can work as designed; then I can offer it to someone else. Such as:
a) a community environmental organisation. (To date: my attempts to make contact with such organisations in Taiwan have proven fruitless; but that was in Kaohsiung; maybe Taichung will be better)
b) a PHD person conducting environmental research. On Loan: not a gift; it cost too much to just give it away.

#73

OK; sometimes things can progress quickly.

0/. Issue with lockup caused by Noise: I2S: Accidentally appear to have fixed this; Not sure if the fix is sustainable; time will tell.
1/. Rationalised reset of Mux at top of AuxBoards:getReading :: Done
2/. SCD4x CO2: I removed atmospheric Pressure compansation call. If that does not do the trick; then I will try a reset to factory config of he sensor; see if it comes to its senses !!

I have also removed pretty much all of the debug output that I consider superfluous. so the UI output is back to sane levels now.

What I see happening:
(a) I see ‘Light’ reading being taken 3 times. Not sure why.
(b) CO2 readings remain zero. I will wait to see if this fixes itself over a longer time frame
(c) I see this:

10:53:35.819 -> (2022-01-21T02:53:33Z) Readings saved to flash memory.
10:53:37.336 -> (2022-01-21T02:53:33Z) 51 readings saved to sdcard.
10:53:40.692 -> Last message repeated 10 times
10:53:44.226 -> Last message repeated 10 times
10:53:47.741 -> Last message repeated 10 times

Its like the host system is not acknowledging reception of readings that I am sending.

When I look at the smartcitizen.me map: readings display; there are anomalies. For example I can no longer scroll right to see readings that do not fit the display. The system seems to think I might have a UV sensor.

I think there is a possibility that the host is not understanding the readings that I am sending. It is also possible I have screwed things up by sending spurious data whilst the system has been undergoing debug.

@oscgonfer I am wondering if you might take a look at the host data for system xrw2e2 (Serial ID 8A76) and ‘clean it up;’ look at the config file: Please tell me what actions I need to take; perhaps amend ID values.
I would like to see the data that I am sending through…

#74

OK; After having written that I had the thought that I should have a look at the docs and see what if any things I can do by myself to facilitate this process.

Thus I have started wading my way through the information given in ‘Handling Calibration Data’ .

I have no intention at this stage of trying to analyse data collected on my SCS locally on my own Laptop.

My goal at this point is simply to select the correct skeleton (if necessary create my own custom skeleton) and related files so that the central system knows what sensors I have installed; and can correctly interpret data that is uploaded.

At the moment thats not the case. My system is spewing out 51 different sensor readings and most of them are inaccessible via the Map.
Downloaded CSV file also does not contain all of the reading values; for example INA219 and SCD4x readings are not there. I have not checked all of them.

#75

Whilst I have been reading through the material learning how to create JSON for device setup on the platform; the system has been going through the first stage of testing.

It has failed that stage. Looks like Mux lockup (again). But after an hour or so. So I have to investigate further and put aside the platform config for a while.

The SCD41 CO2 sensor is finally giving up some reading values. like 15281 ppm.
If CO2 levels were that high I would be very uncomfortable … gasping likely. I opened the window and it dropped by a couple of thousand PPM; but I still do not believe the CO2 level is so high.
Unfortunately its not so easy to get published data on this parameter; I guess its politically sensitive.
All that anyone ever quotes are relative values, not absolute ppm values (the reason I wanted to have this sensor working). But I expected it to tell me levels like 500 ppm…

I cannot put aside the feeling that this thing is fighting me all the way…

#76

OK,

Moving on…
So Today after Sunday break; I tackle some of the issues I observe.

One issue is this:

09:57:01.259 -> Returned from taking a reading from Light
09:57:01.259 -> Light: 7 Lux
09:57:01.259 -> Base::updateSensors :: Reading sensor Noise dBA
09:57:01.259 -> Base::getReading Noise dBA
09:57:01.259 -> Reading requested from Noise dBA
09:57:01.259 -> sckNoise:getReading
09:57:01.259 -> sckNoise:getReading priming of I2S port
09:57:01.517 -> sckNoise:getReading finished priming of I2S port
09:57:01.562 -> sckNoise:getReading finished priming of I2S port
09:57:01.562 -> sckNoise:getReading finished priming of I2S port
09:57:01.562 -> Sck_Noise:getReading: All done
09:57:01.562 -> Returned from taking a reading from Noise dBA
09:57:01.562 -> Noise dBA: 35.94 dBA

This output is produced by this code (just a snippet):

uint32_t startPoint = millis();

	base->sckOut("sckNoise:getReading",PRIO_MED,true);

	//if (!I2S.begin(I2S_PHILIPS_MODE, sampleRate, 32)) return false; // moved to ::start

	// Wait 263000 I2s cycles or 85 ms at 441000 hz
	base->sckOut("sckNoise:getReading priming of I2S port",PRIO_MED,true);		
	
	while (millis() - startPoint < 250) {
			I2S.read();
	}
	
	base->sckOut("sckNoise:getReading finished priming of I2S port",PRIO_MED,true);

Do You see 3 instances of the last line of the code ? No, neither do I.
Do you see anything like ‘GoTo’ in the code; that might repeat that last line of code ? Neither do I.

I cannot; for the life of me understand how or why the system would execute that last line of code 3 times without also executing preceding lines of code 3 times. This is beyond my experience; Everything I have ever learned about coding tells me this stuff is pretty much deterministic.

This tells me something is going on that I do not understand.

I will play with it a little and try to see whats going on.

#77

OK:

I had previously disabled a mechanism in the SckBase::sckOut function that stopped repetitions of the same debug appearing in the output. <-primary cause of this issue

However as to the reason why this particular debug gets repeated remains beyond me. It has obviously been a concern to previous developers (eg fablabsbcn team) otherwise why put in the mechanism in sckOut.

In any case its not worth spending any more of my time on it.

Moving on to other more pressing issues

#78

I determined to see whats happening with my 2nd OLED display.

I had put in some code that should have ensured that 2nd OLed display mirrored the first. But the 2nd OLED remains blank.

I connected the 2nd OLED to a seeeduino V4 (arduino clone) and compliled the example “hello seed” example.
The display remained blank. The example code uses u8g2 library; and the code does not tell the library about the I2C address. But there is a function that does so.
So I included that function: u8g2.setI2CAddress(0x3d);

The display remained blank.

I connected up OLED 1 (set to address 0x3c) and recompiled the code (changing the abovesetI2CAddress) appropriately; and this first OLED worked just fine.

I recall testing this previously; and it worked at that time.

So, I have formed the interim conclusion that perhaps the 2nd OLED has gone faulty.

I have to work out if its worthwhile to order a replacement from Seeed or Mouser or DigiKey. I am leaning in the direction of NOT replacing it;

Moving on to the next issue: why does the system hang immediately after I enter new config details (for wifi connection). When I reset the system after that, it works.
The system did not previously exhibit this behaviour; so something I have done has changed its behaviour.

#79

OK:

Moving on to the next issue : why does the system hang immediately after I enter new config details (for wifi connection). When I reset the system after that, it works.

So, I have put in quite a lot of debug to trace the program flow after entering in new config details.
I have determined that in fact lockup occurs EVERY TIME config command is executed.
I have traced program flow through to the point where the config is saved into eeprom; and ESP is rebooted to pick up the new config details.
At this point program flow stops: specifically after returning from the ‘In()’ function.

I am unsure exactly what is supposed to happen at this point; but from what I have previously observed; the change of state caused by execution of config command causes the system to start reading sensors; compiling the readings and sending them to the platform.

What I think MIGHT BE happening is that because there is a mismatch between what my SCS system is sending and what the platform thinks I have got; this transfer has become problematic; and something is breaking.

It is clear that I need to pay some attention to the platform json data; and I will need some help from the @pral2a1 team to do this, perhaps @oscgonfer perhaps @victor

I will again study the materials provided to help me understand what is going on.
From what I have read to date; there is a json file that describes the skeleton of my system; and its the first jigsaw puzzle piece that would join my SCS to the data it sends through. And right there is where the issue begins; because my SCS is pretty much unique, and there are sensors it knows nothing about (such as SCD41; wind, rain for a start).

I will study some more and see where it leads me.

#80

OK; Still not finished going through the documentation.

My initial impression is that it is not written at all well. There are a lot of words; but written by someone that already understands what is going on. I do not; and have no broad picture in my mind of how it hangs together. The documentation goes some way towards filling the knowledge gap; but there is a lot of space where there is need to interpolate.

What is missing is a task oriented list of things to do; and it would benefit from an Entity-Relationship type of diagram that shows the relationship between various blocks of (json) data.
Starting at the ‘sensors’h’ files (containing the ID values of each sensor as used in the Readings sent through to the platform.)

The documentation refers to a yellow sticker on the cover of the SCS enclosure; indicating the hardware configuration Identifier. Well my kit does not have one of those. Instead I have only the 4 digit serial number of the kit; appearing in several places. I guess it makes some sense. The work to generate a matching skeleton file has not yet been done; as my kit is unique; having sensors not found on other kits.

So, I guess my first step is to generate a candidate for a skeleton file. I can store it on my own github repository. : which is what I will do next (unless something else occurs to me)