With more data created in the last couple years than in the rest of human history combined, the need to manage, manipulate, and secure it has never been more critical. Databases have evolved to keep pace with the growing need, changing to accommodate new ways of gathering and using information or becoming outdated and going the way of the floppy disk. Their future looks even more turbulent as new technologies and ways of interacting with data come into play.
This article outlines five current database trends that explain the booming market for them and offer some idea about what to expect as they continue to evolve with changing technology.
1. Old Guard Losing Out to Cloud DBs
Not so long ago, Oracle, IBM, SAP, Teradata, and Software AG were the bigwigs of the database world. They all began life as on-premises systems and all have attempted to transition to the cloud, with varying degrees of success. However, cloud-based databases have largely taken over and cloud-native databases dominate the market. Microsoft is now the leader, with Amazon Web Services (AWS), Google Cloud Platform (GCP), and Alibaba Cloud close behind. Oracle, IBM, and SAP retain a large slice of the market after a painful transition to cloud-based systems, but cloud is king without question.
Learn more about cloud vs. on-premises architecture.
2. Artificial Intelligence in Databases
On average, database administrators (DBAs) spend 90 percent of their time on maintenance tasks, according to Oracle’s Cloud Business Group surveys. AI is being added to database management as a way to greatly lower the maintenance burden. When well-integrated with databases and their underlying infrastructure, AI helps DBAs spot storage and memory bottlenecks and other issues that inhibit database operations.
3. In-Memory Databases
Today’s mission-critical software solutions require minimal database latency for optimal performance. Unfortunately, traditional database management systems (DBMS) rely on sluggish disk read/write operations for storing data on media (e.g., hard disk drives). For this reason, in-memory databases—databases that store entire datasets in read only memory (RAM)—have become strong alternatives for these critical use cases. Records stored and retrieved directly to and from RAM make faster, more reliable performance possible. Additionally, popular solutions such as Redis—an in-memory data structure store—make it possible for databases to support more data structure types and custom access patterns, allowing for the simplification of software code without data structure conversion or serialization.
4. All-Flash Databases
Memory-based databases are great, but can be very expensive. All-flash arrays provide similar performance at a better price, while also providing a lot more capacity. As a result, more databases now run inside all-flash arrays than on in-memory systems. An example of this is JP Morgan Chase, which was seeing a 30 percent increase or more in data storage needs annually. Greg Johnson, executive director of Global Electronic Trading Services, transitioned from disk-based systems to all-flash arrays to provide the capacity and speed his databases need for transactional and other mission-critical systems. “The combination of all-flash and AI has helped us to approve over 200 million credit card transactions that would have otherwise been declined,” Johnson said.
5. Stronger Database Security Layers
With cyber attacks and data breaches continuing to dominate headlines in the technology world, more focus has been placed on securing the data layer of the software application. In turn, more vendors are augmenting their offerings with stronger built-in security features. Oracle now integrates always-on encryption and automated patching at the database level, for example, while Amazon RDS includes a built-in firewall for rules-based database access. Similarly, database users need far more safeguards related to privacy, data residency, sovereignty, and localization, and DBAs must pay attention to where data is stored and where it is going. Vendors are now introducing location-tracking features into their storage arrays and databases to make it possible to verify compliance.
Learn more about big data security.
Bottom Line: Database Trends
Most databases fall into one of two categories, relational database management systems (RDBMS) and unstructured/special application databases. RDBMS have been around since the 1970s and consist of related tables made up of rows and columns. They’re manipulated using structured query language (SQL), the de-facto standard language for performing create, read, update, and delete (CRUD) functions. This is the dominant database type for enterprise computing.
The advent of the cloud saw data processing capabilities scale horizontally like never before. This happened just in time to support the increase in data generated by the internet—both structured and unstructured. But as unstructured data became increasingly common, a need for a new database paradigm led to the creation of NoSQL, a broad category of databases that do not use SQL as their main language. Because NoSQL databases have no set requirements in terms of schemas or structure, they are ideal for software environments based on DevOps toolsets and continuous improvement/continuous delivery (CI/CD) pipelines.
Technologies come and go, and databases are no different. Early DBAs cut their teeth on Informix, SQL server, and Oracle database management systems, while the next generation favored the simplicity of open-source MySQL/LAMP stack and PostgreSQL. Current DevOps workflows benefit from the unstructured agility of NoSQL databases like MongoDB and DynamoDB.
Where databases go from here will depend upon a number of factors, including technology and market innovations, but the need for them will only continue to increase.
Read next: Top 6 Database Challenges and Solutions