Apache Content Delivery Track

[ Tracks | Register and Attend ]
Wednesday 14:10 UTC
A Test Harness to Verify Traffic Router Performance
Zach Hoffman

As the client-facing entrypoint of an Apache Traffic Control CDN, each Traffic Router must respond to clients quickly and rapidly, but ATC has historically lacked load tests for Traffic Router. The new Traffic Router Test Harness generates a high volume of requests to Delivery Services to ensure that the Traffic Routers' performance meets acceptable thresholds. This talk showcases features and coverage of the Traffic Router Test Harness and demonstrates how a CDN operator can use it to verify that Traffic Router performance meets expectations.

Zach Hoffman lives in Denver, Colorado, is a software engineer at Comcast, and spends spare time contributing to online puzzle game communities and playing the piano. When working on Apache Traffic Control, Zach focuses on its Go and Java components.

Wednesday 15:00 UTC
CDN Locks: A way to ensure data integrity in the CDN
Srijeet Chatterjee

Currently, there is no way to guarantee that only your changes will make their way into a snapshot for a CDN. If somebody else makes changes while you're in the middle of modifying your CDN components or properties, their changes will "dirty" your snapshot and change the integrity of the data that you'd expect in your snapshot. This could result in data mismatches and inconsistencies.
With CDN locks, you can ensure one or both of the following conditions:

  • Only you can make changes and snap/ queue your CDN (exclusive locks)
  • Anybody can make changes but only you can snap/ queue your CDN (shared locks)

Srijeet Chatterjee is a senior software engineer at Comcast, working on the Traffic Control (CDN) team. I've been with Comcast for 5 years, and with the TC team for 1 year and 6 months. Before this, I was working with Echostar (now Dish) for about 3 years. I've been involved with the media/ content field for more than 8 years now, and love every bit of it.
My interests are developing large scale, robust software, and provide a highly satisfactory end product for the customer.
Outside work, I'm a big sports fan, both playing and watching. I love playing cricket and football (soccer) and spend most of my weekends on the field.

Wednesday 15:50 UTC
Traffic Monitor Distributed Polling
Dave Neuman

Traffic Monitor is undergoing a pretty major refactor that change it from only being able to monitor a whole CDN, to being able to monitor part of a CDN if so desired. This talk will discuss the motivation to make these changes, the desired outcome, and the current progress of this effort.

Dave is an Apache member and the former chair of the ATC project. For his day job, Dave is the director of the CDN Application team at Comcast.

Wednesday 17:10 UTC
Updates on CDN Environment Automation and Managing a CDN Operating System at Scale
Jonathan Gray

In the previous two years at ApacheCon North America, we’ve discussed leveraging Ansible to automate the deployment of CDN environments. This talk is provide updates we’ve made this year as well as to discuss how to maintaining CDN operating systems at scale.

Jonathan Gray has been with Comcast on the Content Delivery Network team for approaching 4 years focusing on Operations and Automation. Prior to that he's served as a software developer, integrator, and devops lead for Milsoft Utility Solutions for 7 years. He holds a Bachelor of Science Degree in Computer Science from Abilene Christian University where he also served as an IT Systems Administrator, Datacenter Administrator, and Virtualization Administrator for over 3 years.

Wednesday 18:00 UTC
Caching Proxy Configuration on the Traffic Control CDN
Robert O Butts

The Apache Traffic Control CDN has recently rewritten its monolithic caching proxy configuration tool in Golang as a modular set of UNIX-style apps. We will look at these tools, how they can be used to apply configuration like the previous traffic_ops_ort.pl tool, and how they interoperate and compose with themselves and other UNIX/Linux apps.

Robert O Butts is a software engineer who works on Apache Traffic Control for the Comcast CDN. Rob is a Principal Engineer at Comcast with a Masters degree in Computer Science focused on Parallel Processing. Rob has worked on nearly every component of the Apache Traffic Control CDN. He was the initial author of the Golang Traffic Monitor, Traffic Ops, and the Grove HTTP Caching Proxy. He is currently working on Traffic Control caching and extending Apache Traffic Server for Traffic Control's needs.

Wednesday 18:50 UTC
Proxy Pains and Benefits Supporting HTTP/2 End to End
Susan Hinrichs

Apache Traffic Server (ATS) proxies HTTP traffic. It has supported HTTP/2 for user agent requests almost since the HTTP/2 specification was finalized, but it does not support HTTP/2 for traffic from the proxy to the origin servers. ATS is not unique among proxy servers in supporting new protocols on the client side first. For reverse proxy deployments, many of the benefits for new proxy deployment are seen from the proxy side. An organization’s origin servers are less likely to immediately move to the bleeding edge protocol versions. For forward proxy deployments the benefits of supporting newer protocols are more obvious, but after time even for reverse proxy deployments the need for supporting newer protocols to origin servers increases.
ATS is in the process of refactoring to support multiple HTTP protocols to origin as well as from the user agent. In this talk, I will present some of the design issues we puzzled through and are still considering in this refactoring. For the different types of proxy deployments there are different constraints on supporting multiple protocols to origin. I will also present some preliminary performance experiences with the HTTP/2 to origin code base.

Susan Hinrichs is a computer engineer who specializes in networking, security, and system development. She is a committer on the Apache Traffic Server project and a member of the Apache Software Foundation. Currently Susan leads efforts with the Edge team at Yahoo and now Verizon Media. Prior to that Susan worked in a variety of positions including teaching computer security at the University of Illinois, freelancing work in networking and security, and a couple start ups. Susan participated in the tech bubble by working with a network security company that got acquired by Cisco.
Susan earned a PhD in Computer Science at Carnegie Mellon University and a BS in Computer Science from the University of Illinois at Urbana-Champaign.

Wednesday 19:40 UTC
Defining the data model of HTTP messages, not just the format
Nicholas J. Car

Current Internet protocols (mainly IETF & W3C standards) provide standard ways describe many aspects of HTTP messages, for example the format, language, cache time, etc. Until recently there has been no standard, or even candidate, to define the data model of messages either requested (by a client) or supplied (by a server).
Content Negotiation by Profile is a dual proposed IETF RFQ & W3C Recommendation. It provides mechanics for requesting and supplying HTTP messages that conform to profiles, which are identified data models.
This extremely simple proposed extension to HTTP Headers has profound implications for the Internet: clients can request resources that meet their content needs, not just formats they can parse. Servers can tell clients what data models they support for representations of resources. The indication of profile conformance can be used for automated content validation too.
This presentation will overview the motivation for these proposed standards and explain their potential impact, if adopted. Demonstration systems and data built since the initiation of these proposals will be shown.

Nicholas is internationally recognised as a thought leader and expert in Data Architecture, Data Systems Design, Linked Data and Semantic technologies.
He is active as a standards editor in the W3C’s Dataset Exchange Working Group and the Open Geospatial Consortium’s GeoSPARQL Standards Working Group and a member of the ISO’s TC 211: Technical Committee for the geographic standards information.
He was the design lead for Australian nationally significant and innovative Linked Data and Semantic Web projects, including the Location Index and the Longitudinal Spine of Government Functions, which are seeing emulation internationally.
Nicholas has a Bachelor of Engineering (Hons) from the University of Sydney. and a Ph.D. (Engineering) from the University of Melbourne.

Thursday 14:10 UTC
The Future of Traffic Portal
Steve Hamrick

Traffic Portal (the Traffic Control UI) was written many years ago in angularjs. With angularjs no longer receiving support at the end of 2021, it is important that we upgrade to the new angular. This has many benefits as over the years the web ecosystem has improved substantially. In this session, we will take a look at what new features angular can bring us such as developer speed/tooling, security improvements, and end user experience improvements (time to first paint, page size, accessibility, etc).

Steve has his BS & MS in Computer Science and has 5 years of experience in professional software development. He started using angularjs in 2015 where he rewrote an enterprise app from jQuery to angular. Steve also has experience with angular 2+, including converting multiple angularjs apps into angular. He currently works on the CDN team at Comcast.

Thursday 15:00 UTC
Cache Invalidation - Refresh vs Refetch
Taylor C. Frey

"There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors."

-- Leon Bambrick
Caches exist at all levels of a web request. These caches could be as low as the multi-level architecture within a CPU, stored on a client's computer by the browser, managed in a CDN, or the many steps in-between. While each of these types of caches are implemented differently they certainly share a common constraint: Caches cannot store everything forever. There are both growth (size) and scale (speed/access) limitations that prevent and force a limited capacity for caches. Because of these limitations, there are many strategies, policies, and algorithms to attempt to automatically manage the content being stored within a cache. However, not all caches can be automatically managed. Sometimes we must force a cache to refresh the content being stored, manually.
As a content provider there are times where you may require cached content stored in a CDN to be re-validated. Validation is a process within the CDN to automatically perform a check against the origin to ensure the content is fresh, most often based on a Time-To-Live value or expiration date. This same logic is used to perform a manual check as well. Apache Traffic Control provides an ability to mark the content as stale, triggering an automatic validation with the origin. Still, this might not be sufficient. A mis-configured HTTP server on the origin could still be supplying a response saying the content is fresh, even though it is not. In this case we need the ability to force the CDN to treat the content as a miss, not just stale. To do this we added refetch (TM, hehe) .
Now within ATC, we have the ability to create an invalidation job that forces a cache miss (not just stale), causing the cache to retrieve the content regardless of whether it is currently storing a previous state or not. For example, a news company wishes to update a story as the details arrive. To ensure the customer is retrieving the latest, most accurate, information, a simple refresh invalidation job may not be sufficient. As mentioned before, the cache policy could be set incorrectly, for example. Administrators can create a refetch invalidation job, which can be interpreted by the cache servers as a miss so that the content must be retrieved from the origin. Now our news stories are brought up to date outside of the cache content policy.

Taylor Frey is a Senior Software Engineer working on Apache Traffic Control for Comcast. Taylor received his Bachelor of Science in Computer Science and Master of Science in Software Engineering from Regis University. Since 2011, he has worked across a variety of technology stacks, including mobile (Android and iOS) and desktop applications (Windows and MacOS), although he primarily works with web technologies, focusing on the server side.