Application Delivery Service Challenges in Microservices-based Applications
Microservices based application architectures have ushered in a new era of simplicity in deploying and managing complex application system infrastructure and to scale them almost limitlessly based on traffic demands. But in doing so they have also brought in a new complexities and challenges for IT administrators, devops and application owners that need to be adequately addressed. The end result is that the application delivery services for these modern application architectures have to be provided in a fundamentally new way.
Evolving Application Architectures
Over the last two decade or so three main trends have changed the way applications are put together and delivered to the consumers of applications.
First, the popularity of the web browser and high speed internet as a delivery mechanism led to clients making use of applications via web browsers and eventually through apps on mobile devices, moving away from a desktop model. This naturally led to a SaaS-based business and operational model providing immense economies of scale. Rapid changes to software became a strategy to maintain competitive edge giving rise to the need for continuous integration and continuous deployment.
Second, virtualization, cloud and software-defined infrastructure have made the art of deploying and putting together complex webscale systems into the realm of software programming and thereby brought in a series of tools and methodologies that make it very simple to create such application infrastructures rapidly. This has given birth to the devops culture, a cross between operations and software development, empowering departments and individual app owners to control specific services they need from the infrastructure without having to wait for an IT approval cycle.
A third and the most consequential trend, facilitated by automating many of the time consuming chores of deploying and interconnecting server clusters efficiently, led to a move away from a “hard to debug and maintain” large monolithic main-frame like applications architectures into larger number of more manageable, easy to debug, distributed micro services.
Microservices are small self-contained independently deployable and scalable processes; often using independent technology stacks and OSs which are communicating with other services or clients using lightweight communication and asynchronous protocols, usually REST APIs with JSON payloads and pub-sub message queuing. Logic is separated from state. State is stored in multiple places, distributed and synchronized to be highly resilient. This has given rise to a service economy where businesses expose APIs to data and processing services for others to make these part of a different application composed out of developers own (micro)services as well as other third part services.
A VISUAL COMPARISON: MONOLITHIC VS. MICROSERVICES APPLICATION ARCHITECTURES
Side Effects of the Microservices Trend
On the flip side, applications that are based on traditional approaches run on fewer VMs compared to the same application when divided into microservices. A typical app with just 3 physical servers: one for PHP, one for Java and one for database, by moving to microservices could potentially have 30 services on perhaps a 12-node cluster! The dynamic scalability of the servers based on traffic loads and replication for service availability adds more VMs. Docker containers and cluster managers, data center OS are addressing this challenge.
As a consequence there is both an increase in API service traffic over the web as well as a tremendous increase in East West traffic compared to North South within a traditional data center. An amazon.com page typically makes between 100-150 APIs for rendering any of its pages, at Netflix (2014 data) microservices make 5 billion API requests per day, of which 99.7% are internal. Similarly for salesforce.com their mid-tier API traffic makes up over 60% of the traffic serviced by their application servers (2013 data).
Interactions between microservices as one invokes another creates traffic related dynamic dependencies which can be much more complex than explicitly visible microservice dependencies derived from networking parameters or references in code. Whenever issues arise, it becomes critical to understand the order in which the service were invoked for that specific transaction. Documenting and/or visualizing the fluid application topology is not straightforward.
Speed of software upgrades provides another challenge. As smaller microservices proliferate with different developer ownerships and with continuous deployment methodologies being used, even when the rate of changes are bi-weekly, the application as a whole could be changing at a much faster rate. It is well known that Amazon, Github, Facebook and others deploy production code multiple times a day. The challenge therefore is to automate the switching of production traffic from an older to a newer release of the product. In fact one has to work with the lifecycle of these microservices which can have a small lifespan for handling transient loads and one time functional needs. AWS Lambda, for example, can create ephemeral services lasting 200ms!
The more distributed and interconnected an application is, the more the need to keep a good tab on monitoring. Metrics, analysis, controller actions and automation are the lifeblood for maintaining such a system. Note that an OS can easily generate over 100+ metrics and every app server instance can generate 50+ metrics every second. This means a typical deployment can easily generate thousands of metrics every sec. Any infrastructures to collect, cleanse, collate, correlate and produce actionable controls from these metrics needs to have abstractions and identifiers, attributions needed to make sense of the data.
Consequences for Application Delivery Services
Traditionally application delivery has been about managing operational aspects of an application deployed inside a data center and provide reliable access to any client to any application from anywhere in the world. It typically provides services such as load balancing, traffic management and security. On virtual and cloud infrastructures they are available as pure software images. However, its view of application and traffic flow patterns are traditional and centered on monolithic three-tiered architectures serving clients primarily outside the datacenter and modelled on a traditional appliance form factor.
The microservice phenomenon has proliferated aggressively and is empowering application owners to take more control of their own application delivery services, typically using open source tools like NginX and HAProxy, rather than using an edge based application delivery controllers (ADCs). The central IT team has usually been focused on the edge devices and managing traffic that’s coming in or going out and less on the north-south mid-tier traffic mostly because of existing architectures, cost and complexity. This trends to use OSS disintermediates the central IT, leading to the emergence of a “shadow IT” groups with their own ad hoc policies.
Application delivery services in this context require a fundamental rethink. It needs an architecture that is highly distributed and scales with infrastructure that operates continuously under dynamic changes that need to be discovered in real time by all stake holders. Traffic management must include ability to support upgrades with automatic traffic transition and roll back. Metrics must be collected from diverse instances and correlated on many dimensions based on how application microservices are defined. The following are some of the key challenges faced by application delivery service for microservices:
- Handling Interconnected dynamical services dependency
- Distributed Traffic Management
- Distributed Security
- Operational Intelligence
- Troubleshooting and application level visibility and analytics
Handling effects of failures in dynamic topologies for both hard and soft failures
The first challenging act of any such app delivery service is to be inserted into a dynamic and distributed app topology. It must orchestrate insertion of the service as a cluster of proxies and stitch them to the infrastructure and networking artifacts such as security groups, scaling groups etc. And given the transient nature of physical resources it has to be based on declaratively describing the application, operationally discovering the topology and the networking parameters as deployed and be able to modify it.
An important point that traditional software ADCs miss out is the need to correlate traffic and load across a distributed cluster of service delivery proxies to properly control the system. The diagram below illustrates this point where an app server instance can receive traffic from many ADCs, and on the other hand an ADC instance is forwarding traffic to many app servers. This means that each ADC must take into account what other ADCs are doing to a specific App server.
ARCHITECTING FOR CLUSTERS OF ADCS AND APPLICATION MICROSERVICES
The challenge then becomes how to have an invariant view of an app and its traffic while basic resources are dynamic and also distributed. To address this we need to work on methods of discovery of these relationships via a registry and notification-based strategies followed by powerful visualization for the system operator; We also need a way to apply policies on the abstract view of the app. Containerized microservice create their own networking related challenges that these systems must address.
Distributed systems have many points of failures with intricate dependencies and some will invariably fail. Due to the asynchronous and dynamically changing interconnections and interactions between many different entities it can be difficult to zero in on hard-to-find failures. Failures can be both “soft” or transient as well as “hard” or final. Worse, simple failures in one component can sometimes trigger a cascade of failures across the interdependent microservices of the app infrastructure. Hard failure recovery must be done dynamically and rapidly – needing signaling of impending failure as well as recovery of failed instances in a timely manner.
The promise of microservices is profound and can lead to significant application agility and increased time to market. Traditional architectures need to rapidly evolve and application delivery mechanisms need to keep in step. In short application services for the microservices age merits fresh thinking and has to be re-architected for changing needs.
Blog post courtesy of Appcito
By Alak Deb
