mainnet network outage with no way to report it

[fluidvoice]

Describe the bug
at ~5AM EST Jan 6 the Stacks blockchain has not received any of the Bitcoin blocks for at least 3 hours. Mempool.space shows there are bitcoin blocks but Stacks is not receiving them.

I could not find any way to report this outage to Stacks Core on the Hiro Systems network status page or other page. Additionally the Hiro Status page was not reporting/reflecting this outage, so there may be another bug in the Hiro service.

Is there a page I just don't know about?...
A way for the public to report outages that trigger notifications to the blockchain core team is badly needed.

Steps To Reproduce
n/a above

Expected behavior
a link on Hiro System's status page where outages can be reported.

Environment (please complete the following information):
n/a

Additional context
This was originally reported by users on the Stacks Chat Telegram channel.

[jcnelson]

We've known since the issue began. There is already monitoring infrastructure in place at each org at which the devs work, and it successfully altered us 10 minutes after the stall.

…

On Mon, Jan 6, 2025, 5:24 AM Brad Baker ***@***.***> wrote: *Describe the bug* at ~5AM EST Jan 6 the Stacks blockchain has not received any of the Bitcoin blocks for at least 3 hours. Mempool.space shows there are bitcoin blocks but Stacks is not receiving them. I could not find any way to report this outage to Stacks Core on the Hiro Systems network status page or other page. Is there a page I just don't know about?... A way for the public to report outages that trigger notifications to the blockchain core team is badly needed. *Steps To Reproduce* n/a above *Expected behavior* a link on Hiro System's status page where outages can be reported. *Environment (please complete the following information):* n/a *Additional context* This was originally reported by users on the Stacks Chat Telegram channel. — Reply to this email directly, view it on GitHub <#5659>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQJK2A6MJWPMVHXIFKYQL2JJKVRAVCNFSM6AAAAABUVKE76CVHI2DSMVQWIX3LMV43ASLTON2WKOZSG43TAMZUGAZDCOA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[fluidvoice]

Thank Jude. is it purposeful to have the Hiro System status page not show the status in real-time? I mean I can think of reasons why it would be designed this way.

[fluidvoice]

Maybe move this issue to Hiro Systems Status Page service as a bug or enhancement request?
Since ideally this public facing service should better reflect the network status.
Additionally, users could see an outage is already known by Stacks Core and being investigated/resolved.

[jcnelson]

We're putting the banner up right now.

[jcnelson]

From what I can tell, the issue is that a couple signers fell offline that put the network under 70% signing power. We've been trying to get ahold of them for the past couple hours.