Back to Smartcitizen.me

Platform stopped Working

Hi morning!

I got a message that my kit wasn’t publishin but after a reset, checking the wifi and confirming LED status are ok (blue-green), realized no other kits were online in the web.

I think the server is down!! :frowning:

My kit stopped publishing 05:58 UK time this morning as well.

Tor

1 Like

The server left for summer holidays then :sweat_smile:

Yes, mine stopped publishing at same hour, 6:58 Barcelona time… maybe tomorrow the smartcitizen guys we’ll be able to fix it

Somebody has to push the reset button in Barca.

Hi, it started with me with Lux (@6:28 AM CET) and @6:58 AM CET everything stopped. I restarted it (even my wifi router). That did not help.

That’s peak holidays here!

I hope my co-citizens were working like me and could push the reset button :sweat_smile:

My case, in Japan, mySCKs stopped working at 13:58, 4th, Aug.

BTW, I applied filter on the website (https://smartcitizen.me/) as it only shows online SCKs. The website showed only 16 kits. Interestingly, the 16 kits are working with SCK 1.1. I don’t know why, but assumingely the problem is on MQTT server?

1 Like

Mine said it had stopped publishing yesterday at 05.59am. When I got back the lights all seemed ok, but I reset it and re paired it with my WiFi. Didn’t make any difference, so I connected via the SDK via USB and saw that it WAS publishing every minute) to somewhere which seemed to be sending back an acknowledgement and also issued the “hello” command which pinged the server and got a positive response, so I will stop fiddling until, what sounds like the server, is sorted out . See below

SCK > PM 1.0: 1 ug/m3
PM 2.5: 1 ug/m3
PM 10.0: 1 ug/m3

(2019-08-05T08:42:20Z) Sent 9 readings to platform.
(2019-08-05T08:42:20Z) Readings saved to sdcard.
(2019-08-05T08:42:20Z) Published OK, erasing from memory

hello
Hello sent!
Waiting for MQTT hello response…
SCK > Hello OK!!

Just to add that I am using the API and get responses from the API for my kit - but they are from 05:58 when the publishing part seems to have stuck.

My kit stopped publishing 06:59 NL on the 4th of aug

I think the kits are publishing to the server (as per the messages I get directly from my kit via the USB console connection) , I just hope the server is storing them and that the problem is that stopping them being accessible though the website and map is fixed and will be come available

This is now getting a bit silly - with automated messages telling me that my kit stopped publishing (it did not - monitoring is just getting it wrong) and not even a single comment (that I can find anyway) from the project team.

I know the whole platform is open source, and it may be time to look at having a working mirror available through a community effort, or some of the reasonably competent users access to debug when outages like this happen.

Any interest in doing something like this?

Anyone from the project team no on holidays?

Any updates?

Regards
Tor

1 Like

Anyone home? I’m posting now through the ‘Feedback’ tab hoping someone from SCK team will notice.

The server left the building and everybody is between Mar Bela and Barceloneta. Canicula.

1 Like

:rofl::rofl::rofl:

Yes, that’s too hot to work here.

You are right, one of the services (MQTT) went down!
We are reviving it as we speak.

Thank you all!

EDIT: The service is back to normal.

1 Like

What I am slightly confused about was that when I looked at the kit via the USB console. it confirmed every minute that it was successfully publishing and that if I did a “hello” commands it also confirmed a successful MQTT transmission and acknowledgement. (see below). How can that be if MQTT was down?

SCK > PM 1.0: 1 ug/m3
PM 2.5: 1 ug/m3
PM 10.0: 1 ug/m3
(2019-08-05T08:42:20Z) Sent 9 readings to platform.
(2019-08-05T08:42:20Z) Readings saved to sdcard.
(2019-08-05T08:42:20Z) Published OK, erasing from memory

hello
Hello sent!
Waiting for MQTT hello response…
SCK > Hello OK!!

Yes, the MQTT server was was up, but we have a small MQTT Subscription service which connects from the API to the MQTT server and subscribes to it. That service stopped working properly.
That service is responsible to pull (subscribe) data from the MQTT server to the Rails API.
This means that kits can still publish data, but the data did not go all the way down the chain.

A simplified data flow is something like this:
SC Kit -> MQTT -> Rails API -> KairosDB -> Cassandra Cluster

the SCK here does not work since July 18 – today is end of Aug 9 on US East Coast .
At least it should have saved everything automatically the moment it disconnected from your server, and go online as soon as your server was up.
Unreliable - is the word.