Background
Apache ShardingSphere is an Apache Top-Level project and is one of the most popular open-source big data projects. It was started about 5 years ago, and now ShardingSphere has over 14K+ stars and 270+ contributors in its community.
The successful project has already launched and updated many versions. Apache ShardingSphere now supports many powerful features and keeps optimizing its configuration rules. We want to help users understand all features and configuration rules, to help them quickly test and run components,and ultimately help them achieve best performance, so we decide to start the shardingsphere-example project.
shardingsphere-example is an independent Maven project. It’s preserved in the “examples” file of Apache ShardingSphere. [Link]:(https://github.com/apache/shardingsphere/tree/master/examples)
Modules & Explanation
The shardingsphere-example project contains many modules. It provides users with guides and configuration examples of features like horizontal scaling, read and write separation, distributed governance, distributed transaction, data encryption, hint manager, shadow database, etc.
It also includes common tools such as Java API, YAML, Spring Boot, Spring Namespace. In addition to ShardingSphere-JDBC, now we add use examples of ShardingSphere-Proxy and ShardingSphere-Parser in shardingsphere-example. You can easily find all features of Apache ShardingSphere as well as their scenarios, and their flexible configurations in our official repo. The table below shows how the modules are distributed in shardingsphere-example.
shardingsphere-example
├── example-core
│ ├── config-utility
│ ├── example-api
│ ├── example-raw-jdbc
│ ├── example-spring-jpa
│ └── example-spring-mybatis
├── shardingsphere-jdbc-example
│ ├── sharding-example
│ │ ├── sharding-raw-jdbc-example
│ │ ├── sharding-spring-boot-jpa-example
│ │ ├── sharding-spring-boot-mybatis-example
│ │ ├── sharding-spring-namespace-jpa-example
│ │ └── sharding-spring-namespace-mybatis-example
│ ├── governance-example
│ │ ├── governance-raw-jdbc-example
│ │ ├── governance-spring-boot-mybatis-example
│ │ └── governance-spring-namespace-mybatis-example
│ ├── transaction-example
│ │ ├── transaction-2pc-xa-atomikos-raw-jdbc-example
│ │ ├── transaction-2pc-xa-bitronix-raw-jdbc-example
│ │ ├── transaction-2pc-xa-narayana-raw-jdbc-example
│ │ ├── transaction-2pc-xa-spring-boot-example
│ │ ├── transaction-2pc-xa-spring-namespace-example
│ │ ├── transaction-base-seata-raw-jdbc-example
│ │ └── transaction-base-seata-spring-boot-example
│ ├── other-feature-example
│ │ ├── encrypt-example
│ │ │ ├── encrypt-raw-jdbc-example
│ │ │ ├── encrypt-spring-boot-mybatis-example
│ │ │ └── encrypt-spring-namespace-mybatis-example
│ │ ├── hint-example
│ │ │ └── hint-raw-jdbc-example
│ │ └── shadow-example
│ │ │ ├── shadow-raw-jdbc-example
│ │ │ ├── shadow-spring-boot-mybatis-example
│ │ │ └── shadow-spring-namespace-mybatis-example
│ ├── extension-example
│ │ └── custom-sharding-algortihm-example
├── shardingsphere-parser-example
├── shardingsphere-proxy-example
│ ├── shardingsphere-proxy-boot-mybatis-example
│ └── shardingsphere-proxy-hint-example
└── src/resources
└── manual_schema.sql
example-core
The module example-core contains entity, interface definition and other public codes
shardingsphere-jdbc-example
The example module ShardingSphere-JDBC displays ShardingSphere-JDBC features and how to use them.
- sharding-example
The module displays how to use ShardingSphere-JDBC to scale out in scenarios like sharding, horizontal scaling, veritical scaling, read and write seperation, as well as read and write seperation plus sharding.
In terms of integration with ORM, this module also provides users with examples of MyBatis and JPA integrations.
- governance-example
This module is about the distributed governance of ShardingSphere-JDBC, and includes related scenerios combined with features like sharding, read and write seperation, data encryption, shadow database.
Note: The example of distributed governance depends on Apache Zookeeper. Please adopt self-deloyment.
- transaction-example
This module displays the multiple ways of distributed transaction management ShardingSphere-JDBC supports. A user can base on his application and choose an appropriate distributed transaction coordinator. Given the complexity of distributed transactions, all examples in this module are based on vertical scaling, horizontal scaling and sharding.
Note: When you use Seata, please adopt self-deployment.
- other-feature-example
This module gives examples of some ShardingSphere-JDBC features, i.e., encrypt (data encryption), hint (hint manager), shadow (shadow database).
- encrypt-example
This module displays examples of data encryption. It also tells users how to use and access Java API, YAML, Spring Boot, Spring Namespace.
- hint-example
This shows examples of hint manager. However, at present, there is only YAML configuration example. We welcome more scenarios.
- shadow-example
This gives examples of shadow database, including its application combined with data encryption, sharding, and read/write separation.
- extension-example
The module tells users how to use custom extension of ShardingSphere-JDBC. Users can leverage SPI or other ways provided by ShardingSphere to extend features.
- custom-sharding-algortihm algorithm-example
The module shows how a user can use CLASS_BASED
and customize his sharding algorithm.
shardingsphere-parser-example
SQLParserEngine
is the SQL parse engine of Apache ShardingSphere. It is also the base of ShardingSphere-JDBC and ShardingSphere-Proxy. When a user inputs a SQL text, SQLParserEngine
parses it and makes it recognizable expressions. Then it’s fine to have enhancement such as routing or rewriting.
Following the release of its 5.0.0-alpha version, Apache ShardingSphere’s core feature SQL Parser is totally open to its users. They can use API and call SQLParserEngine
. This way they can meet more of their own business demands by having such an effective SQL parsing in their systems.
In the module, users can learn how to use SQLParserEngine
API. It provides different syntactical rules of languages, such as MySQL, PostgreSQL, Oracle, SQL Server and SQL 92.
shardingsphere-proxy-example
The example module of ShardingSphere-Proxy includes configuration examples of common scenarios like sharding, read and write separation and hint manager. Since features of ShardingSphere-Proxy are almost the same as that of ShardingSphere-JDBC, users can refer to shardingsphere-jdbc-example
when they fail to find the example they want in shardingsphere-proxy-example.
- shardingsphere-proxy-boot-mybatis-example
In the module users can learn how they can use Proxy to configure sharding, and how they use SpringBoot + MyBatis to access data.
- shardingsphere-proxy-hint-example
In this module, a user can know how to use Proxy to configure hint manager and how to use Java cliend-end to access data.
New Optimization
Apache ShardingSphere 5.0.0-beta version is coming soon, so the community contributors also have updated shardingsphere-example. They optimized the following:
JDK version
Component version
ClassName
Configuration profiles
SQL script
Related details are as follows:
JDK version upgrade
According to JetBrains’s “A Picture of Java in 2020”, Java8 LTS is the most popular version among Java developers.
Following this update, shardingsphere-example uses Java 8 and newer versions. If you use Java 7 or earlier versions, please update your JDK version first.
String dependency upgrade
In shardingsphere-example, we update string dependency components.
• spring-boot version from1.5.17 to 2.0.9.RELEASE
• springframework version from 4.3.20.RELEASE to 5.0.13.RELEASE
• mybatis-spring-boot-start version from 1.3.0 to 2.0.1
• mybatis-spring version from 1.3.0 to 2.0.1
Persistence framework upgrade
In sharding-sphere-example, we update the persistence frameworks MyBatis and Hibernate.
• mybatis version from 3.4.2 to 3.5.1
• hibernate version from 4.3.11.Final to 5.2.18.Final
Connection pooling upgrade
In sharding-sphere-example, we update the database connection pool HikariCP.
• HikariCP artifactId from HikariCP-java7 to HikariCP
• HikariCP version from 2.4.11 to 3.4.2
Database driver upgrade
In sharding-sphere-example, we update the database connection drivers of MySQL and PostgreSQL
• mysql-connector-java version from 5.1.42 to 5.1.47
• postgresql version from 42.2.5.jre7 to 42.2.5
Example
In this section, we give several typical examples and show you how to configure and run shardingsphere-example.
There are many modules in the project shardingsphere-example. But for now, we only choose several popular application scenarios of ShardingSphere-JDBC.
Preparation
Maven is the project’s build tool of shardingsphere-example. Please prepare for it first;
Prepare Apache ShardingSphere. If you have not downloaded Apache ShardingSphere, please download and compile it first. You can use the reference below:
git clone https://github.com/apache/shardingsphere.git
cd shardingsphere
mvn clean install -Prelease
Import the shardingsphere-example project to your IDE;
Prepare a manageable database environment, such as local MySQL examples;
If you need to test read and write separation, please make sure that your your master-slave synchronization is OK;
Execute the DB init script:examples/src/resources/manual_schema.sql
Scenarios & Examples
- sharding-spring-boot-mybatis-example: Sharding
1. Path
examples/shardingsphere-jdbc-example/sharding-example/sharding-spring-boot-mybatis-example
2. Goal
This example shows the application of using ShardingSphere-JDBC in combination with SpringBoot and MyBatis to realize sharding. The sharding goal is to shard one table into four evenly preserved in two different databases.
3. Preparation
Configure application.properties
set spring.profiles.active as
sharding-databases-tables
Configure application-sharding-databases-tables.
Change
jdbc-url
with your database location and setup your user ID, password, etc.Set the attribute of
spring.shardingsphere.props.sql-show
astrue
See more details in Configuration Manual
4. Run
Run at startup:ShardingSpringBootMybatisExample.java
Now, you can observe routing of all SQL expressions in Logic SQL and Actual SQL of logs and understand how sharding works.
Sharding-raw-jdbc-example: Read and write splitting
1. Path
examples/shardingsphere-jdbc-example/sharding-example/sharding-raw-jdbc-example
2. Goal
The example shows how users can use YAML and configure the feature read and write splitting of ShardingSphere-JDBC . The goal is to separate one writing database and two reading databases.
3. Preparation
Configure META-INF/readwrite-splitting.yaml
Change
jdbc-url
with your database location and setup your user ID, password, etc.Set
props.sql-show
astrue
See more details in Configuration Manual.
4. Run
Open the startup: ShardingRawYamlConfigurationExample.java
and set shardingType
as ShardingType.READWRITE_SPLITTING
. Run the startup.
Now, you can observe routing of all SQL expressions in Logic SQL and Actual SQL of logs and understand how read and write splitting works.
Note: when master-slave database synchronization fails, there will be query errors.
Custom-sharding-algortihm-example: Custom algorithm
- Path
examples/shardingsphere-jdbc-example/extension-example/custom-sharding-algortihm-example/class-based-sharding-algorithm-example
- Goal
The example shows how users can use CLASS_BASED
and extend their own custom algorithm . By doing so, ShardingSphere-JDBC can base on a custom algorithm and calculate sharding results during sharding routing. The scenario is how to use custom sharding algorithm to scale out.
- Preparation
Prepare your sharding algorithm that shall base on business needs and use any interfaces of
StandardShardingAlgorithm
,ComplexKeysShardingAlgorithm
orHintShardingAlgorithm
. In the example, we show how to useClassBasedStandardShardingAlgorithmFixture
Configure META-INF/sharding-databases.yaml
Change
jdbc-url
with your database location and setup your user ID, password, etc.Set
props.sql-show
astrue
.
Note: For shardingAlgorithms
, when a type is CLASS_BASED
, you can use props
and assign class and absolute path of a custom algorithm. Thus, the configuration is done.
See more details in Configuration Manual.
4. Run
Run the startup: YamlClassBasedShardingAlgorithmExample.java
Now you can use logs and observe your database. You can also use methods like DEBUG and check input and output of your custom algorithm.
Summary
Our brief ends here. In the future, we will share with you more examples of ShardingSphere-JDBC, ShardingSphere-Proxy and ShardingSphere-Parser.
If you have any questions or have found any issues, we are looking forward to your comments on our GitHub issue, or you can submit your pull request and join us, or join our Slack community. We welcome anyone who would like to be part of this Top-Level project and make a contribution. For more information please visit our Contributor Guide.
Top comments (0)