Database Upgrade
Scheduled Maintenance Report for dYdX
Completed
The scheduled maintenance has been completed.
Posted Oct 26, 2021 - 18:22 UTC
Update
The matching engine is back online and the metrics look better now. Verifying all fixes/upgrades
Posted Oct 26, 2021 - 18:14 UTC
Update
Redeploying our matching-engine (which doesn't rely on the database) which seemed to be having some latency issues processing messages
Posted Oct 26, 2021 - 18:11 UTC
Update
We are still seeing very slightly elevated latency in round-trip processing time for orders (on the order of 200ms). Will update when resolved
Posted Oct 26, 2021 - 18:01 UTC
Update
The upgrade looks successful but Cleanup is still ongoing. We will update here when latency looks normal again
Posted Oct 26, 2021 - 17:36 UTC
Verifying
Cleanup started. Summary Below:

10-15 Minutes: Cleanup
- Orders and cancelations should start to proceed as normal again
- We expect a small amount of slowness as the new database instance pages-in entries to its cache and background jobs tune the database tables
Posted Oct 26, 2021 - 17:12 UTC
Update
Failover started. Summary Below:

1-2 minutes: Failover and Service Restarts
- The start time of this step is unknown and is triggered by the end of the database provisioning and upgrade mentioned above
- The old instance will be taken down and new services will be changed to point to the new database instance.
- Placing or canceling of orders will fail during this time
- Triggerable (i.e. Stop-Limit or Take-Profit) orders will not trigger
- Most other API requests will return 500 errors at this time
Posted Oct 26, 2021 - 17:11 UTC
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Oct 26, 2021 - 17:00 UTC
Update
Upgrade step started. Summary Below:

5-15 minutes: Upgrade
- A new, upgraded database instance will be provisioned in the background
- No service interruption or degradation at this time
Posted Oct 26, 2021 - 17:00 UTC
Update
Maintenance is still scheduled. A rough timeline of steps and what to expect starting at 17:00 UTC. The maintenance will be updated as we move through these steps.

5-15 minutes: Upgrade
- A new, upgraded database instance will be provisioned in the background
- No service interruption or degradation at this time

1-2 minutes: Failover and Service Restarts
- The start time of this step is unknown and is triggered by the end of the database provisioning and upgrade mentioned above
- The old instance will be taken down and new services will be changed to point to the new database instance.
- Placing or canceling of orders will fail during this time
- Triggerable (i.e. Stop-Limit or Take-Profit) orders will not trigger
- Most other API requests will return 500 errors at this time

10-15 Minutes: Cleanup
- Orders and cancelations should start to proceed as normal again
- We expect a small amount of slowness as the new database instance pages-in entries to its cache and background jobs tune the database tables

Afterwards: Completion
- Faster response times across the board when placing/canceling orders, especially during peak activity
Posted Oct 25, 2021 - 20:55 UTC
Update
We have moved the planned-maintenance day

From: Thursday, October 21
To: Tuesday, October 26
Posted Oct 20, 2021 - 19:02 UTC
Scheduled
We will be updating our database during this time. We expect about 2 minutes of downtime and possibly some general slowness.
Posted Oct 18, 2021 - 02:35 UTC
This scheduled maintenance affected: trade.dydx.exchange.