Why is my database "unreachable"?
Our monitoring system opens and maintains a single connection to your Memetria K/V database so that it can track the health of your server. This monitoring system is responsible for graphs, alerts, and performance monitoring. When it is unable to communicate with your server, it records the problem on your graphs. If the problem is fatal, it will also send an alert to on-call Memetria engineers.
If you click on an orange block in your graphs, you'll see a popup that explains when and why your server was unreachable. Below, you'll find more information about what these different messages mean:
Timeouts
Timeouts occur when our monitoring system is connected to your server but does not receive a response from your server within a specified interval of time (currently set at 2 seconds). This can happen for a number of reasons:
- Long-running commands. Redis is single-threaded, which
means that it can only run one command at a time. If you have commands that
take a long time to run (popular culprits are O(n) commands like
KEYSandSMEMBERS), they can cause your server to become backed up and cause timeouts. You can find a list of slow commands on your server's dashboard under the "Summary" and "Slow Queries" tabs. - Too many commands. If you have a large number of clients connected to your server, it's possible (though uncommon) to send commands at a rate that your server simply can't keep up with, causing a backlog of commands to build up. If you think that you're experiencing this problem, send us an email at support@memetria.com and we'll help you out.
- Transient networking issues. Timeouts can also be caused by temporary networking problems such as congestion or partitions. If you're experiencing frequent networking issues with your server, send us an email at support@memetria.com.
Maximum number of clients reached
Memetria K/V is configured to accept a maximum number of clients to protect the performance of your database. When this threshold is reached, your server will begin rejecting new connections. If you are seeing "maxclients" errors frequently, it can indicate a problem with your client code. Send an email to support@memetria.com and we'll work with you to fix the problem.
Unable to connect
If our monitoring system is unable to establish a connection with your server entirely, it will immediately send an alert to Memetria engineers, who will investigate and solve the problem as quickly as possible. Connection errors have a number of potential causes:
- Hardware failure. If the hardware underlying your server fails, we will immediately create a new server with your most recent backup. Once this backup server is up and running, we'll contact you to help you manage your failover.
- Networking failure. In some cases, a server may still be running but unreachable from the outside world due to networking failures. If this occurs, we'll create a recovery server and work with you to fail over to it.
- Software crash. Although exceedingly rare, it's not impossible K/V to crash. If this occurs, your server will automatically restart and recover from its AOF file, which provides the most up-to-date version of your database.
Other problems
If nothing above answers your questions or if you're seeing problems not listed here, send us an email at support@memetria.com and a Memetria engineer will help you.