io errors on aws ephemeral storage volume

PIOPs is an optional but critical feature of EBS that gives your volume reliable and predictable performance. Consistent reliable IO performance is critically important in maintaining the performance of the database. When using EBS volumes in support of a MongoDB deployment, we advise using Provisioned IOPs (PIOPs). Using Elastic Block Store: Avoid Noisy Neighbors! Also, use replica set tagging to isolate the node from production load. In that case be sure to configure this secondary with a priority of 0 to prevent this instance becoming a primary. Secondaries configured in this way are often used exclusively for backups and may not be capable of handling a full combined load of reads and writes. Even in the case that you've configured a replica set using ephemeral storage, you may configure that at least one of the secondaries in the set use EBS volumes as an added assurance of data persistence, just be sure that the EBS volumes are provisioned to match the write load of the primary lest they fall behind in replication. While such a deployment may not require high database availability itself, an instance failure shouldn't mean that you've unnecessarily lost your data as well. Ephemeral storage isn’t suited for a stand alone deployment of MongoDB. This doesn't mean that EBS volumes aren't valuable. As long as your MongoDB deployment is running as a properly configured replica set, distributed across multiple availability zones, the risk of losing data from an individual instance failure is mitigated. Fortunately, MongoDB provides high availability through replication. However, when an instance is terminated the ephemeral drive and its data is lost. Instance stores have lower latencies since the requests do not need to go over the network, and several instance types come with multiple attached volumes that can be RAIDed for greater throughput and capacity. They have better performance, decreased cost, and decreased complexity. That is, if the EC2 instance on which MongoDB is running should fail, the EBS volume and the data it contains is recoverable.ĮC2 instance stores (as known as ephemeral storage) are disks that are physically attached to the host instance. EBS volumes are replicated within their availability zone and are durable if the EC2 instance to which they are attached should fail or terminate. Elastic Block Store or Instance Store? Or, both?ĮBS volumes are persistent block-level storage volumes that are attached to EC2 instances. For the purposes of this post, we're going to focus on the factors that matter when deciding between Amazon's two storage options: Amazon Elastic Block Store (EBS) versus EC2 Instance Store. Selecting which combination of these strategies is best depends on your specific workload and their respective costs.

Increase the capacity of the database by sharding (effectively increasing RAM and IO bandwidth as an aggregate across the set of sharded nodes).

Increase the IO bandwidth of the individual instance (e.g.

Upgrading the instance type to one with more RAM.

You may either scale your system to meet the load by one, or a combination of, the following options. This can lead to saturated disk and severe performance degradation. If your working set exceeds available memory, documents will repeatedly be read from disk. In some applications this can be a small fraction of your total database, whereas in others it may actually be your entire database.

The working set is the portion of your data that is frequently accessed. The document is retained in the cache once it has been loaded, and subsequent operations on the document will be much faster as access to disk won't be necessary.Ī critical factor in the performance of the system is understanding the size of your application’s working set. Any request for a document that is not currently in memory is first read from disk and loaded into memory. That is, for a document to be accessed that document must be present in the in-memory cache.

MongoDB processes operations on data in memory. So, what better to talk about than how to select high performance storage for running MongoDB on EC2? A Brief Refresher The excitement around AWS re:Invent put us in a blogging mood.

Io errors on aws ephemeral storage volume