“Getting a little bit of the right information just ahead of when it’s needed is a lot more valuable than all the information in the world a month or a day later.” That quote – found in the book The Two Second Advantage by Vivek Ranadive and Kevin Maney – highlights a new reality where responsiveness can be a competitive advantage. Smart companies are building a responsive IT infrastructure where data isn’t just hoarded in massive repositories, but analyzed quickly and acted upon. How can you know more, faster and have better situational awareness?
With an increasing amount of critical IT systems running in the cloud, there’s a need to know what’s happening and act on it. This month, CenturyLink Cloud introduced Webhooks, making us among the first public IaaS cloud providers to send real-time notifications to a web service endpoint. For this initial release, customers can set up Webhooks for events within accounts, users, and servers.
When To Use This?
Webhooks are relatively new idea, although already used by diverse web properties like Wordpress and Zoho. Let’s look at three different scenarios where CenturyLink Cloud Webhooks can lead to better decisions.
Scenario #1 – Data Synchronization
Polling is an inefficient way to retrieve data from an external system, but it remains a popular choice. When you poll a system for changes, you’re effectively asking “do you have anything new for me?” Many times, the answer is “no.” With push-based notifications, the only time you are contacted is when something relevant happens. For example, some customers synchronize CenturyLink Cloud data with their internal support or configuration management systems. They do this for auditing purposes, or to give support staff an accurate picture of cloud deployments. The issue? Staying in sync requires an aggressive polling frequency that needless encumbers systems. Webhooks provide a better alternative.
In the scenario visualized below, as soon as a new server is created in the CenturyLink Cloud cloud, an event fires and a message is sent to an endpoint specified by the customer. That listener service then updates the appropriate internal system. Within seconds, systems are completely synchronized!
Scenario #2 – Anomaly Detection
People love the cloud because of the self-service capabilities and freedom to instantly create and delete servers at will. One downside of this freedom – for service providers anyway – is fraudulent signups. CenturyLink Cloud resellers actively monitor new accounts, but the sheer volume of manual analysis can be daunting. What if resellers could programmatically monitor specific sequences of events and then use that data to flag an account as “suspect” and deserving of special attention? Again, we turn to Webhooks to help react faster.
It’s great that developers can quickly bring gobs of new cloud machines online. But rapid provisioning can occur within the wrong sub-account or under unusual circumstances. In both of these examples, consider using a complex event processing solution that monitors streams of Webhook events and detects aggregate patterns that reveal more than any single event can.
Scenario #3 – Compliance Monitoring
Cloud and governance don’t have to be at odds with each other – and in fact, these two ideas go hand-and-hand when it comes to IT as a service. CenturyLink Cloud already provides customers with many ways to do this today through sophisticated account management capabilities. But we often get customers requesting a “corner case” scenario – like preventing a certain user from being added to an account, or making sure that database servers aren’t given a public IP address. Webhooks are a way for us to programmatically empower customers to support unique scenarios, in self-service fashion. Via Webhooks, users compare events to previous ones using a data repository. This way, customers can immediately find out if a server was changed inappropriately, a user was added to an account, or the contact information was changed. If an out-of-compliance change is made, the customer can respond almost instantly!
It’s very simple to configure Webhooks in the CenturyLink Cloud cloud. Simply visit the API section of the Control Portal and choose Webhooks. Here, users can browse the list of available Webhooks, then specify the “target” URL to receive a JSON-encoded message. Each Webhook is configured with an HTTPS URL, and includes an optional capability to send events that occur within sub-accounts.
For more details on how to create a Webhook listener service, take a look at our Webhook FAQ article in the Knowledge Base. This is an innovative and exciting capability for the platform and we can’t wait to see how customers use it to create more responsive systems and processes!
Elasticity is a core tenet of cloud computing. Cloud has become so popular simply because resources can be adjusted up or down, based on business need, instantly. Manually resizing cloud environments is still MUCH easier than altering physical hardware. But human action is still required, adding human cost to cloud.
A few cloud vendors have attempted to automate this process through “auto scaling” – services that expand and reduce the size environments based on user-defined parameters. However, this capability by and large automates the addition and removal of virtual machines to an existing resource pool. In engineering terms, this is “horizontal scaling” – adding capacity across multiple virtual machines. This approach is useful for consumer applications (think Netflix scaling up for Saturday night), but the enterprise scenario is much different, as we found out in our market research when developing this feature.
While we always recommend that our customers build highly available cloud systems with no single points of failure, there is value is sizing those resources up and down (i.e. “vertical scaling”) instead of only being able to add or remove entire servers. Having multiple servers is key for fault tolerance, but some workloads can benefit from additional server capacity, not just more servers!
This month, CenturyLink Cloud introduced our new Autoscale service. The initial release is focused on vertical scaling of CPU resources, with more vertical scaling (and, yes, horizontal scaling!) on the roadmap. Today, you can now add and subtract CPUs from cloud servers based on user-defined utilization limits. Capacity is added instantly without a reboot and capacity is removed only during user-defined windows of time, to prevent a reboot from occurring during prime usage hours.
We generate massive amounts of data every day. Research firm IDC estimates that 90% of the world’s data was created in the last two years, and the volume of data worldwide doubles every two years. Enterprises are a key contributor to this data explosion as we produce and share digital media, create global systems that collect and generate data, and retain an increasing number of backup and archive data sets. This rapid storage growth puts pressure on IT budgets and staff who have to constantly find and allocate more usable space. CenturyLink Cloud wants to help make that easier and just launched a new Object Storage service to provide you a secure, scalable destination for business data.
What is Object Storage from CenturyLink Cloud? It’s a geo-redundant, elastic storage system for public and private digital data. Based on the innovative Riak CS Enterprise platform, Object Storage infrastructure is being deployed across three global regions: Canada, United States, and Europe. Each region consists of a pair of CenturyLink Cloud data centers that run Riak CS Enterprise on powerful, bare-metal servers. The Object Storage nodes are deployed in a “ring” configuration where data is evenly distributed across the nodes, thus assuring that your data is available even if multiple nodes go offline. When objects are loaded into one data center, they are instantly replicated to the in-country peer data center. This means that an entire data center can go offline, and you STILL will have uninterrupted access to all of your latest enterprise data.
Before diving into this new service, let’s define a few terms:
- Object. An “object” is any digital asset that is less than 5 GB in size. This could be a video that you display on your public website, a PDF file that you are sharing with a business partner, or a database backup file. If the object is larger than 5 GB, then you can do a multi-part upload!
- Bucket. Objects are stored in buckets. A bucket is a logical container that can hold an unlimited number of objects, but not other buckets.
- Region. CenturyLink Cloud has architected Object Storage with unique clusters in three different geographies. Each geographic region has a pair of data centers that hold all of the data uploaded into that region.
- User. An Object Storage user is different from a CenturyLink Cloud platform user and is created separately. While you may create an Object Storage user to represent an individual person, you may also choose to create users that correspond to an application. For example, you may define a user leveraged by your public website that retrieves images and videos from Object Storage.
- Owner. Each bucket has an owner. This is the user that automatically has full control over the bucket and its objects.
- ACLs. Access Control Lists govern who can manage buckets and see objects. By default, Object Storage does not allow any public access to buckets or objects. If you choose, you can provide public, unauthenticated users with the ability to read individual objects. Or, you can choose specific users that have permission to add objects to buckets or view an object.
Managing Object Storage
Interacting with Object Storage is easy. We’ve added a management interface in our Control Portal for Object Storage administrators. From here, you can view a list of users, add new users, and reset user credentials.
The Control Portal also has a bucket administration component where you can view, create, secure, and delete buckets.
Each bucket can have its own security profile. For a bucket such as “website media”, you may let “All Users” have read access to its objects. For buckets set up to exchange large files with business partners, you would likely add read and write permissions for a user representing the chosen partner.
It’s unlikely that you’ll only use a single interface to interact with your data objects. Thanks to the inherent S3 compatibility offered by Riak CS Enterprise, you don’t have to! There is an entire ecosystem of tools for working with object storage that support an Amazon S3-like interface. Want to use a client tool to upload and delete objects? Then check out a utility like the freemium S3 Browser where you can plug in your Object Storage user credentials (and CenturyLink Cloud Object Storage URL) and manage buckets AND objects.
Looking to mount Object Storage as a drive on your database server so that you can easily create and restore backups? Look to a product like ExpanDrive which makes it easy to add Object Storage as a storage volume.
CenturyLink Cloud is among the first cloud providers to offer native, geo-redundant object storage and we’re excited to see how our customers use this to escape the burden of endless provisioning of on-premises storage! Our Canada region is live today, with the United States and Europe following closely. Existing customers can get started right away, and new customers can take Object Storage for a spin by signing up today.
While the cloud has become a welcome channel for companies refining their server footprint, it’s still rare to find an established business that is running *solely* in the cloud. Rather, many companies leverage the cloud for specific workloads and new cloud-first applications, while keeping other servers and applications in-house. But are you tracking your cloud servers the same way you track on-premises ones? Do you have a single place to see a list of ALL your servers and when they last changed? Configuration management databases (CMDBs) are a popular way to store information about IT assets such as who owns them, where they physically are, and their change history. How can you take advantage of the cloud while retaining a complete, up-to-date CMDB? One option is to programmatically link cloud servers to your CMDB through the use of cloud APIs. In this blog post, we’ll see an example of that process in action.
Step 1: Link Cloud Servers to CMDB Entries
Let us first consider the “IT-as-a-Service” scenario where an internal customer portal serves as the launching pad for new cloud servers. Using the CenturyLink Cloud API, customers can easily provision and manage their cloud servers without ever logging into our Control Portal.
Here, the customer’s own portal gives internal employees the opportunity to quickly spin up a cloud server. After adding a record to the CMDB and getting back the CMDB record locator, the CenturyLink Cloud CreateServer API operation is called. CenturyLink Cloud servers can have user-defined metadata attached to them, and in this case, that metadata consists of the CMDB record ID. The server build request is queued by the CenturyLink Cloud engine and the name of the new server is returned by calls to the GetDeploymentStatus API operation. The name of the CenturyLink Cloud server can optionally be added to the CMDB configuration item in order to create a bi-directional link between the systems. At this point, the internal CMDB has a list of servers built internally or in the CenturyLink Cloud cloud.
Step 2: Synchronize Updates
A wonderful aspect of the cloud is the ease by which someone can create, modify, and destroy servers on demand. This means that you do not want to get stuck manually maintaining records of cloud servers that are constantly in flux. Inevitably, the effort to keep the CMDB up to date will fail and it becomes an unreliable record of IT asset configurations. How can you easily synchronize your CMDB with CenturyLink Cloud? Use the APIs!
CenturyLink Cloud’s Engineering team just added new API operations that make it simple to retrieve a list of all servers that have changed within a certain period of time. Customers can run a simple application every evening and invoke the GetAllServersByModifiedDate API operation to pull back a list of all CenturyLink Cloud servers that have experienced the following events:
- Created or deleted
- Paused/powered on/powered off/reset/rebooted/shut down
- CPU count, RAM amount, storage amount changed
- Public IP added or released
- Snapshot created/restored/deleted
- Archived or restored from archive
- Custom (metadata) field added, or value changed
- Software installed or script executed (via Blueprint)
Most of these changes are extremely relevant to a configuration database and provide critical context about the cloud server’s lifecycle. By automating these changes with the API, you can save significant administration time and effort.
CMDBs are a critical component for many enterprises, and your cloud servers should be a visible part of your IT asset management strategy. CenturyLink Cloud is constantly working to deliver a powerful API that provide the glue to connect your on-premises systems and cloud resources. Existing customers have instant access to our API today and new customers can get started by signing up for an account today!
It’s easy for cloud customers to get confused about the roles and responsibilities of their internal team and their cloud vendor. That confusion is especially evident when it comes to application availability and business continuity planning. How does disaster recovery differ from high availability? Does my cloud provider automatically load balance my application servers? The answers to these questions are critical, but sometimes overlooked until a crisis occurs. In this post, we’ll talk about load balancing, high availability, and disaster recovery in the cloud, and what the CenturyLink Cloud’s cloud infrastructure has to offer.
What is it?
Wikipedia describes load balancing as:
Load balancing is a computer networking method to distribute workload across multiple computers or a computer cluster, network links, central processing units, disk drives, or other resources, to achieve optimal resource utilization, maximize throughput, minimize response time, and avoid overload. Using multiple components with load balancing, instead of a single component, may increase reliability through redundancy.
You commonly see this technique employed in web applications where multiple web servers work together to handle inbound traffic. There are at least two reasons why load balancing is employed:
- The required capacity is too large for a single machine. When running processes that consume a large amount of system resources (e.g. CPU and memory), it often makes sense to employ multiple servers to distribute the work instead of constantly adding capacity to a single server. In plenty of cases, it’s not even possible to allocate enough memory or CPU to a single machine to handle all of the work! Load balancing across multiple servers makes it possible to host high traffic websites or run complex data processing jobs that demand more resources than a single server can deliver.
- Looking for more reliability and flexibility in a solution deployment. Even if you *could* run an entire server application on a single server, it may not be a good idea. Load balancing can increase reliability by providing many servers able to do the same job. If one server becomes unavailable, the others can simply pick up the additional work until a new server comes online. Software updates become easier since a server can simply be taken out of the load balancing pool when a patch or reboot is necessary. Load balancing gives system administrators more flexibility in maintaining servers without negatively impacting the application as a whole.
Load balancing can be accomplished using either a “push” or a “pull” model. For web applications or database clusters that sit behind a load balancer, inbound requests are pushed to the pool of servers based on an algorithm such as round-robin. In this scenario, servers await traffic sent to them by the load balancer. It’s also possible to use a “pull” model where work requests are added to a centralized “queue” and a collection of servers retrieve those requests from that queue when they are available. For instance, consider big data processing scenarios where many servers work to analyze data and return results. Each server takes a chunk of work and the overall processing load is distributed across many machines.
How can CenturyLink Cloud help?
CenturyLink Cloud offers multiple load balancing options to our customers. All customers have access to a free, shared load balancer. This load balancer service – based on the powerful Citrix Netscaler product – provides a range of capabilities including SSL offloading for higher performance, session persistence (known as “sticky sessions”), and routing of TCP, HTTP and HTTPS traffic for up to three servers. To use this service today, send a request to firstname.lastname@example.org. We plan to launch a self-service version of this capability in the very near future.
If you’re looking for more control over the load balancing configuration or have higher bandwidth needs, you can deploy a dedicated load balancer (virtual appliance) into the CenturyLink Cloud cloud. This “bring your own load balancer” option leverage internal expertise you may have with a particular vendor. It also gives you complete control over the load balancer setup so that you can modify the routing algorithm or enable/disable features that matter to your business.
What is it?
Returning to Wikipedia, high availability is defined as:
High availability is a system design approach and associated service implementation that ensures a prearranged level of operational performance will be met during a contractual measurement period.
High availability is described through service level agreements and achieved through an architecture that focuses on constant availability even in the face of failures at any level of the system. While load balancing introduces redundancy, it’s not a strategy that alone can provide high availability. Servers sitting behind a load balancer may be running, but that doesn’t mean that they are available!
Availability addresses the ability to withstand failure from all angles including the network, storage, and even the data center itself. Enterprise cloud services like those from CenturyLink Cloud are built on a highly available architecture that uses redundancy at all levels to ensure that no single component failure in a data center impacts overall system availability. This includes “passive” redundancy built into data centers to overcome power or internet provider failures, as well as “active” redundancy that leverages sophisticated monitoring to detect issues and initiate failover procedures.
All of our customers get platform-level high availability when they use the CenturyLink Cloud cloud “out of the box.” That means that you can rely on us for your workloads knowing that our architecture is well-designed and highly redundant. However – back to the introductory paragraph – it’s the customer’s responsibility to design a highly-available application architecture. Simply deploying an application to our cloud doesn’t make it highly available. For example, if you deploy a single Microsoft SQL Server instance in the CenturyLink Cloud cloud, you do not have a highly available database. If that database server goes offline or network access is interrupted, your application’s availability will be impacted. To design a highly available Microsoft SQL Server solution, you have multiple options. One choice is to create a cluster of database servers (where all nodes are active at the same time, or, nodes sit passively by waiting to be engaged) that access data from a shared disk. When a failure in the active node is detected, the alternate node is automatically called into action.
How can CenturyLink Cloud help?
Designing highly available systems is complex. Unfortunately, no cloud provider can offer a checkbox labeled “Make this application highly available!” in their cloud management portal. Crafting a highly available system involves a methodical approach that navigates through every single layer of the system and identifies single points of failure that should be made redundant. For components that cannot be made redundant, it’s important to make sure that the application can continue to run even if that component becomes unavailable.
The CenturyLink Cloud professional services team consists of skilled, experienced architects who have designed and built cloud-scale solutions for customers. They can sit with your team and make sure that you’ve taken advantage of every relevant feature that CenturyLink Cloud has to offer, while helping you make sure that your system landscape is constructed in a way that will ensure continual availability.
Don’t forget to regularly test your high availability design in order to uncover weak points or ensure that configurations remain valid.
What is it?
Once more we turn to Wikipedia which defines disaster recovery as:
Disaster recovery (DR) is the process, policies and procedures that are related to preparing for recovery or continuation of technology infrastructure which are vital to an organization after a natural or human-induced disaster. Disaster recovery is a subset of business continuity. While business continuity involves planning for keeping all aspects of a business functioning in the midst of disruptive events, disaster recovery focuses on the IT or technology systems that support business functions.
DR is all about how you handle unexpected events. Typically, your cloud provider has to declare a disaster before explicitly initiating DR procedures. A brief network outage or storage failure in a data center is usually not enough to trigger a disaster response. There are two phrases that you often hear when defining a DR plan. A recovery point objective (RPO) describes the maximum window of data that can be lost because of a disaster. For example, an RPO of 12 hours means that it is possible that when you get back online after a disaster, you may have lost the most recent 12 hours of data collected by your systems. A recovery time objective (RTO) identifies how long the IT systems (and processes) can be offline before being restored. For example, an RTO of 48 hours means that it may take two days before the systems lost in the disaster are brought back online and becoming usable again.
How can CenturyLink Cloud help?
CenturyLink Cloud customers have disaster protection natively in the platform. We offer two classes of storage: standard and premium. The major difference is that standard storage get five days of rolling backups within a given data center, while premium storage users get fourteen days of rolling backups including replication to an in-country data center. CenturyLink Cloud is powered by global data centers in multiple countries and we use storage replication to enable you to get back online within 8 hours (RTO) and with a maximum RPO of 24 hours.
While this provides assurances against losing all of your data in the event of a disaster, it still may not provide the level of business continuity that you need. If your business cannot tolerate more than a few moments of downtime, even in the event of a disaster, then it’s critical to architect a solution that can withstand the loss of an entire data center. Returning to our earlier Microsoft SQL Server example, consider the ways to construct a highly available database that remains online with minimal data loss, even during a disaster. SQL Server offers replication technologies like database mirroring and AlwaysOn that make it possible to do near-real time replication across geographies.
The experts in the CenturyLink Cloud services team can help you identify all the DNS, networking, compute and storage considerations for building systems that are not only highly available within a data center, but across data centers.
It’s often the case that load balancing, high availability and disaster recovery lapses don’t surface until it’s too late. While CenturyLink Cloud does everything we can to architect our platform for maximum availability and resiliency, our customers still retain responsibility for deploying their systems in a manner that meets their performance and business continuity needs. We are eager to talk to you about how to validate your existing cloud applications or design new solutions that can function at cloud scale. Contact our services team today!