What Is Sharding in MongoDB And How it Works?
Sharing is an algorithm for dispersing information across multiple computers, allowing horizontal scaling (instead of vertical scaling).
Vertical scaling is the process of improving the performance of a single device or single server with an enhanced CPU, increased RAM, or an increase in storage capacity.
If physical limitations weren’t an issue, vertical scaling is preferred because of its ease of use. In the real world, no single machine can handle the demands of modern workloads.
Horizontal scaling is the solution. Also called scale-out, it’s the act of adding nodes that divide the data set and load. Horizontal scaling allows near-infinite scaling to handle vast volumes of data and heavy workloads.
How Does Sharding Work?
Shards, Routers, and Config Servers
Within MongoDB it is a Sharded-Cluster is composed of many:
- Shards
- routers
- Replica sets for the config server
The term “shard” refers to a replica set that is one subset of a collection (also called chunks). A replica set is made up of one or more mongod nodes that share the same data set.
Replica sets are highly available. Sharding lets us partition the data over multiple replica sets and consequently increase the resources of databases.
A router(or mongos)acts as a query router for applications that use clients that handle writes and read operations. The router stores the metadata of the config server to ensure that it can guide requests to the correct Shard or Shards.
The config servers keep the entire metadata of a sharded group. The metadata includes the state and organization of every component and data in the sharded cluster.
In its most basic configuration, using one shard, a cluster that is sharded could be like this:
Source: https://www.mongodb.com/basics/sharding
This configuration isn’t yet ready for production, but it’s suitable in a test environment. For production environments, we strongly suggest using multiple shards/mongos routers:
Source: https://www.mongodb.com/basics/sharding
What are the Benefits of Sharding?
Sharding lets you scale the database you use to manage increasing demands to an almost infinite extent. It is accomplished by increasing the amount of data that can be read and written, storage capacity, and your database’s availability. Let’s examine each of them in more in detail:
More efficient read and write speed: by distributing the data set over several shards, both capacities for write and read operations can be increased if these operations are limited to one shard.
It is contingent on the way you define the key for shards in each of the sharded collections.
Storage capacity increased: Additionally, in growing the number of shards, you can increase the total capacity of storage. It will allow for near-infinite scaling.
High availability: Because every shard is a replica set, each piece that is stored is replicated. Because all data are distributed when one shard fails, the entire database is still functional to reading and writing from all the remaining shards.
Shard Keys and Chunks
Shard Keys
Sharing a collection is a process that requires the user to choose the method of distribution of the collection’s contents among different shards. MongoDB makes use of its shard keys to spread the collection’s content across shards. It is done by assigning a set of values to the shard.
Fields within every document determine shard keys. The fields’ values will pick the shard on which the document will be placed according to the shard ranges and the number of chunks. The information is then stored in the replica of the config server set.
https://www.mongodb.com/basics/sharding
Every shard key must have an index or auto-build as part of the sharding commands if the collection is incomplete and the necessary index isn’t present.
The key that shards are stored in can directly affect the cluster’s performance and can cause problems with your application’s access patterns. Utilize the following information to determine the correct key for your shard.
Chunks
Chunks are the logical categories of documents that are moved across shards according to the sharding range mappings at a particular moment in time. Each chunk comprises an inclusive lower range and an exclusive upper range based on the key for shards.
When a chunk reaches an exact size, or when the workload calls for it, the chunk is cut into two. The balanced component will automatically distribute the chunks between shards in order so that they remain as evenly as they can be.
How to Implement Sharding
The most cost-effective, most convenient, and easiest way to manage and deploy a sharded cluster is through MongoDB Atlas, the Database-as-a-Service that clarifies sharded cluster implementation.
All you have to be able to do is to turn on:
https://www.mongodb.com/basics/sharding
Why Use MongoDB Atlas?
- Install your sharded cluster within minutes using just a few clicks. It lets you split your collection across different regions and cloud providers and create one cluster to cover the global network.
- Scale in and out with one command using unified monitoring tools and performance advisory.
- It’s affordable since it costs only on your Shards infrastructure, not for configuration routers or servers.
- It comes with built-in security for enterprise and encryption for access to your sharded cluster and auditing.
- Updates to application that are no downtime and OS platform patching is done for you.
If you are ready to built it by your own follow these steps. It would help if you replaced everything between < and > with the appropriate values for your configuration. We’ll set up the config server and shard(s) using replica sets with three nodes and a mongos server.
Let’s start.
Set Up the Config Server
Each replica set of config servers can include the number of mongod processes (up to 50) but with the following restrictions: no arbiters and no members with zero priority. In each case the above, you’ll need to begin the process using an option called the option –configsvr option. For instance:
mongod –configsvr –replSet <configReplSetName> –dbpath <path> –port 27019 –bind_ip localhost,<hostname(s)|ip address(es)>
Then, integrate with solely one process of the replica set.
mongo –host <hostname> –port 27019
Then execute rs.initiate() on only one of the members of the replica set:
rs.initiate(
{
_id: “<configReplSetName>”,
configsvr: true,
members: [
{ _id : 0, host : “<cfg1.example.net:27019>” },
{ _id : 1, host : “<cfg2.example.net:27019>” },
{ _id : 2, host : “<cfg3.example.net:27019>” }
]
}
)
Once the replica of the config server is in place and running, it is possible to create shards.
Set Up Shards
As previously mentioned, each shard can be described as a copy on its own. This procedure is identical to the config servers; however, using this –shardsvr option. Be sure to select an alternate name for each replica sets.
mongod –shardsvr –replSet <shardReplicaSetNamereplSetname> –dbpath <path> –port 27018 –bind_ip <cluster hostname(s)|ip address(es)>
Then, integrate through solely one process of the replica set:
mongo –host <hostname> –port 27018
Then, run rs.initiate() on solely one process of the replica set. Be sure to remove the — configsvr option:
rs.initiate(
{
_id: “<shardReplicaSetNamereplSetnamereplSetName>”,
members: [
{ _id : 0, host : “<shard-host1.example.net:2701827019>” },
{ _id : 1, host : “<shard-host2.example.net:2701827019>” },
{ _id : 2, host : “<shard-host3.example.net:2701827019>” }
]
}
)
Begin with mongos
Lastly, configure mongos and direct them to the replica servers of your config server:
mongos –configdb
<configReplSetName>/<cfg1.example.net:27019>,<cfg2.example.net:27019>,<cfg3.example.net:27019> –bind_ip localhost,<cluster hostname(s)|ip address(es)>
In a production environment, more than one mongo is required to prevent congestion and ensure availability. Typically, beginning with at least three mongo instances is a good idea.
Configure and Turn On Sharding for the Database
Connect your Mongos to:
mongo –host <hostname> –port 27017
Add your shards to the cluster. Make this one time per shard.
sh.addShard( “<shardReplicaSetName>/<shard-host1.example.net:27018>,<shard-host2.example.net:27018>,<shard-host3.example.net:27018>”)
Sharding can be enabled on your database:
sh.enableSharding(“<database>”)
Last but not least, make sure you shard your collection with your sh.shardCollection() technique. It can be done by the hashed sharding method, which spreads your data equally across shreds.
sh.shardCollection(“<database>.<collection>”, { <shard key field> : “hashed” , … } )
Or through range-based sharding, which permits the optimal distribution of data between shards according to key values of the shard. For specific sets of information, it may allow queries to be made across a variety of information more effective. The command is like this:
sh.shardCollection(“<database>.<collection>”, { <shard key field> : 1, … } )
That’s it! Now you’ve set up your first cluster that is sharded. Every interaction with your application from now on must be handled via routers (mongos instances) only.
If you want to read more please click here
