Segments are now processing in realtime, and all systems are operating normally. This incident is resolved.
Posted Apr 01, 2024 - 19:30 EDT
Monitoring
Segments referencing profile properties have reduced to less than 25 minutes delay, and expect to be back to realtime within the hour. We will continue to monitor until the issue is resolved.
Posted Apr 01, 2024 - 18:12 EDT
Update
The team is continuing to see improvements, and requests to the Identity API are no longer delayed. Segments referencing profile properties continue to be delayed by up to 4 hours.
Posted Apr 01, 2024 - 17:16 EDT
Update
The team has made improvements, and event processing is no longer delayed.
Segments referencing profile properties continue to be delayed by up to 4 hours. Requests to the Identify API are delayed by 1 hour. We are still working to resolve these delays.
Posted Apr 01, 2024 - 16:55 EDT
Update
The team has made improvements to address the underlying issue, but we continue to observe delays. Segments referencing profile properties are currently delayed by up to 4 hours. Requests to the Identify API and event processing are both delayed by 1 hour.
We will continue to work to resolve the delays.
Posted Apr 01, 2024 - 16:09 EDT
Update
The team is triaging this issue and has pushed initial fixes. Segments referencing profile properties continue to be delayed by up to 2.5 hours. Requests to the Identify API continue to be delayed by 1 hour.
Through further investigation we’ve also discovered that event processing is also delayed by up to 50 mins. We continue to work to fix the underlying issue and will keep this page updated with further developments.
Posted Apr 01, 2024 - 14:18 EDT
Identified
Starting at 11:20am (US Eastern), any segment referencing a profile property in the criteria has been delayed in processing. Additionally, processing requests to the Identify API (e.g. profile creation & updates from onsite tracking) is delayed by 1.5 hours. Any Flows using these segments will also be impacted. The team is currently working to triage the issue, and we do not expect any lost data once these delays resolve.