阿里巴巴大数据 —玩家社区 关注
手机版
  1. 云栖社区>
  2. 阿里巴巴大数据 —玩家社区>
  3. 博客>
  4. 正文

Alibaba Cloud MaxCompute vs. AWS Redshift vs. Azure SQL Data Warehouse

隐林 2018-01-05 19:38:14 浏览240 评论0 发表于: 阿里巴巴大数据 —玩家社区 >> MaxCompute(原ODPS)

Cloud SQL Redshift database IT alibaba service MaxCompute

摘要: Data is the currency of the digital world. How your organization stores, organizes, analyzes, and uses the data within its confines will largely determine how successful it is.

6e06294dbfcb4956edf52e3310e5b481fd6d07f0_jpeg

Data is the currency of the digital world. How your organization stores, organizes, analyzes, and uses the data within its confines will largely determine how successful it is. Enterprises deal with large quantities of data, typically at petabyte scale, and they look to glean maximum value from all this data.

Cloud computing has been a game changer in this respect. What would be cost-prohibitive with traditional servers is now much more accessible with the economic and powerful solutions offered by cloud computing vendors.

Case in point: Data warehouse solutions hosted completely in the cloud. Thanks to cloud-based data lakes, what would have been impossible a few years ago is now made possible by the plummeting costs of data storage disks, and more powerful compute instances. This post explains how to use data warehouses in the cloud, and compares popular options on major public cloud platforms.

Data Warehouse Basics

A data warehouse is a centralized data store that’s used by multiple applications within your organization. If you’re looking to analyze small quantities of data that are a couple of GB in size, a data warehouse is too complex for your needs. A data warehouse makes sense only once you’ve scaled to a few hundred GB of data. At that point, you can’t function at the same speed and agility you used to, and you need a data warehouse.

The first thing to know about a data warehouse is that it is architected differently from small-scale database infrastructure. Rather than having databases that are restricted to hardware servers, a data warehouse is made of multiple servers that work together as a single unit.

Alibaba Cloud MaxCompute

Alibaba Cloud’s MaxCompute is a large-scale data analysis platform that is purpose-built for running big data workloads. Coming from the house of Alibaba, it was built out of necessity, when Alibaba needed a way to manage their ever-growing data that Oracle servers could no longer handle. It is battle-tested internally at Alibaba, where it has run a cluster of 10,000 server nodes. On a daily basis, 14,000 developers at Alibaba run three million jobs on it, and it stores 99% of all of Alibaba’s data. It is the first database service to scale to 100 TB of data at 7,000 BigBench Query-per-minute (BBQpm).

MaxCompute makes data migration simple with a variety of options. You can use Alibaba Cloud’s own tools like the MaxCompute client, or DataWorks, or even popular external tools like Flume, Logstash, or Fluentd. The uploaded data is stored in an SQL database, and can easily be scaled up to petabytes in size.

The most recent version of MaxCompute supports SQL 2.0, and interestingly allows for querying of unstructured data like images and video content. Despite the large quantities of data, and some of it being unstructured, MaxCompute is especially well-suited for real-time analysis. And the best part is that it is extremely easy to use and maintain. MaxCompute handles the difficulty of managing a distributed data store by having unique processes for clustering, indexing, and join optimization which all help with better data storage and retrieval at large scale.

With its recent US launch, MaxCompute is ready to change the way Big Data is processed across the world. With aggressive pricing, it is ready to take on similar services from the two other big cloud vendors—AWS and Azure.

AWS Redshift

AWS Redshift is one of the early services from the AWS stable. Similar to MaxCompute, it stores and analyzes data at petabyte scale. You can load data into Redshift using many AWS services such as S3, DynamoDB, or an SSH-enabled host on EC2. It leverages AWS IAM for security and access permissions. Further, you can encrypt your data using KMS, either on server-side or within the AWS cloud.

A unique feature of the service is Redshift Spectrum, which lets you query data that’s already in AWS S3. This means you don’t have to load your data into Redshift or transform your data. Instead, you can get to querying the data directly. However, if you’d rather have your data in Redshift and you have a lot of it, AWS Glue is an ETL service that makes data loading easy.

AWS recently announced new DC2 nodes which replace DC1 nodes at the same cost. They’re based on Intel’s Broadwell chips and offer twice the performance of the previous DC1 nodes and 30% better storage utilization.

With a variety of options for usage, AWS Redshift is an attractive option for data warehousing in the cloud.

Azure SQL Data Warehouse

Azure SQL data warehouse is the Big Data analysis solution from Microsoft. With Microsoft’s big footprint among the Fortune 500 enterprises, many of its customers would be interested in this service. Azure provides two flavors of this service—one optimized for elasticity, and the other optimized for compute. You could separate workloads across these two tiers, and it makes for an interesting choice. You can allocate or measure usage in the form of Data Warehouse Units (DWUs). There are two types of DWUs—a regular DWU, and a cDWU which is optimized for compute. Azure provides a level of service for each database which is measured in the form of Database Transaction Units (DTUs).

Azure has a tool called PolyBase, which is used to query external data without requiring the user to know Hadoop. PolyBase lets you import and export data to and from Hadoop, Azure Blob Storage, or Azure Data Lake Store, or query the data without moving it in and out of SQL Data Warehouse. SQL Data Warehouse is also well integrated with PowerShell, which lets you use scripting to automate common tasks.

Conclusion

In conclusion, all three data warehouse services mentioned here are powerful tools that take a different approach to the same challenge—analyzing big data in real time. If you have broader commitments that require you to choose Redshift or SQL Data Warehouse, it’s not a bad spot to be in. However, if you’re curious to try a powerful new option that is also cost-effective, MaxCompute is the way to go. Alibaba Cloud is offering a $300 credit for new users, making it easy to get a feel for what the platform has to offer before going all in. Try MaxCompute and start unlocking value from all your data in real time.

Bio

Twain Taylor

Twain began his career at Google, where, among other things, he was involved in technical support for the AdWords team. His work involved reviewing stack traces, and resolving issues affecting both customers and the Support team, and handling escalations. Later, he built branded social media applications, and automation scripts to help startups better manage their marketing operations. Today, as a technology journalist he helps IT magazines, and startups change the way teams build and ship applications

如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件至:yqgroup@service.aliyun.com 进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容。

用云栖社区APP,舒服~

【云栖快讯】中办国办印发《推进互联网协议第六版(IPv6)规模部署行动计划》加快推进基于 IPv6 的下一代互联网规模部署,计划指出2025年末中国 IPv6 规模要达到世界第一,阿里云也第一时间宣布了将全面提供IPv6服务,那么在全面部署 IPV6 前,你需要了解都在这儿  详情请点击

网友评论