This talk is a concise masterclass on how to write infrastructure code. I share key lessons from the “Infrastructure Cookbook” we developed at Gruntwork while creating and maintaining a library of over 300,000 lines of infrastructure code that’s used in production by hundreds of companies. Come and hear our war stories, laugh about all the mistakes we’ve made along the way, and learn what Terraform, Packer, Docker, and Go look like in the wild.
A comprehensive walkthrough of how to manage infrastructure-as-code using Terraform. This presentation includes an introduction to Terraform, a discussion of how to manage Terraform state, how to use Terraform modules, an overview of best practices (e.g. isolation, versioning, loops, if-statements), and a list of gotchas to look out for.
For a written and more in-depth version of this presentation, check out the "Comprehensive Guide to Terraform" blog post series: https://blog.gruntwork.io/a-comprehensive-guide-to-terraform-b3d32832baca
This document discusses Terraform, an open source tool for building, changing, and versioning infrastructure safely and efficiently. It provides declarative configuration files to manage networks, virtual machines, containers, and other infrastructure resources. The document introduces Terraform and how it works, provides examples of Terraform code and its output, and offers best practices for using Terraform including separating infrastructure code from application code, using modules, and managing state. Terraform allows infrastructure to be treated as code, provides a faster development cycle than other tools like CloudFormation, and helps promote a devOps culture.
This document contains notes from a talk on advanced Terraform techniques. It discusses using Terraform for infrastructure as code to deploy resources across multiple environments like development, staging, and production. It also mentions techniques like separating code into modules, using variables to parameterize configurations, and integrating Terraform with other DevOps tools like Ansible.
This document introduces infrastructure as code (IaC) using Terraform and provides examples of deploying infrastructure on AWS including:
- A single EC2 instance
- A single web server
- A cluster of web servers using an Auto Scaling Group
- Adding a load balancer using an Elastic Load Balancer
It also discusses Terraform concepts and syntax like variables, resources, outputs, and interpolation. The target audience is people who deploy infrastructure on AWS or other clouds.
This document provides an overview of Terraform including its key features and how to install, configure, and use Terraform to deploy infrastructure on AWS. It covers topics such as creating EC2 instances and other AWS resources with Terraform, using variables, outputs, and provisioners, implementing modules and workspaces, and managing the Terraform state.
The document provides an overview of Terraform and discusses why it was chosen over other infrastructure as code tools. It outlines an agenda covering Terraform installation, configuration, and use of data sources and resources to build example infrastructure including a VCN, internet gateway, subnets, and how to taint and destroy resources. The live demo then walks through setting up Terraform and using it to provision example OCI resources.
A comprehensive walkthrough of how to manage infrastructure-as-code using Terraform. This presentation includes an introduction to Terraform, a discussion of how to manage Terraform state, how to use Terraform modules, an overview of best practices (e.g. isolation, versioning, loops, if-statements), and a list of gotchas to look out for.
For a written and more in-depth version of this presentation, check out the "Comprehensive Guide to Terraform" blog post series: https://blog.gruntwork.io/a-comprehensive-guide-to-terraform-b3d32832baca
This document discusses Terraform, an open source tool for building, changing, and versioning infrastructure safely and efficiently. It provides declarative configuration files to manage networks, virtual machines, containers, and other infrastructure resources. The document introduces Terraform and how it works, provides examples of Terraform code and its output, and offers best practices for using Terraform including separating infrastructure code from application code, using modules, and managing state. Terraform allows infrastructure to be treated as code, provides a faster development cycle than other tools like CloudFormation, and helps promote a devOps culture.
This document contains notes from a talk on advanced Terraform techniques. It discusses using Terraform for infrastructure as code to deploy resources across multiple environments like development, staging, and production. It also mentions techniques like separating code into modules, using variables to parameterize configurations, and integrating Terraform with other DevOps tools like Ansible.
This document introduces infrastructure as code (IaC) using Terraform and provides examples of deploying infrastructure on AWS including:
- A single EC2 instance
- A single web server
- A cluster of web servers using an Auto Scaling Group
- Adding a load balancer using an Elastic Load Balancer
It also discusses Terraform concepts and syntax like variables, resources, outputs, and interpolation. The target audience is people who deploy infrastructure on AWS or other clouds.
This document provides an overview of Terraform including its key features and how to install, configure, and use Terraform to deploy infrastructure on AWS. It covers topics such as creating EC2 instances and other AWS resources with Terraform, using variables, outputs, and provisioners, implementing modules and workspaces, and managing the Terraform state.
The document provides an overview of Terraform and discusses why it was chosen over other infrastructure as code tools. It outlines an agenda covering Terraform installation, configuration, and use of data sources and resources to build example infrastructure including a VCN, internet gateway, subnets, and how to taint and destroy resources. The live demo then walks through setting up Terraform and using it to provision example OCI resources.
This document provides an overview and introduction to Terraform, including:
- Terraform is an open-source tool for building, changing, and versioning infrastructure safely and efficiently across multiple cloud providers and custom solutions.
- It discusses how Terraform compares to other tools like CloudFormation, Puppet, Chef, etc. and highlights some key Terraform facts like its versioning, community, and issue tracking on GitHub.
- The document provides instructions on getting started with Terraform by installing it and describes some common Terraform commands like apply, plan, and refresh.
- Finally, it briefly outlines some key Terraform features and example use cases like cloud app setup, multi
This document provides an overview and agenda for a Docker networking deep dive presentation. The presentation covers key concepts in Docker networking including libnetwork, the Container Networking Model (CNM), multi-host networking capabilities, service discovery, load balancing, and new features in Docker 1.12 like routing mesh and secured control/data planes. The agenda demonstrates Docker networking use cases like default bridge networks, user-defined bridge networks, and overlay networks. It also covers networking drivers, Docker 1.12 swarm mode networking functionality, and how concepts like routing mesh and load balancing work.
This document summarizes a meetup about infrastructure as code. It discusses the differences between treating infrastructure as "pets" versus "cattle", where pets are cared for individually and cattle are treated as disposable. When infrastructure is coded declaratively using tools like Terraform, the infrastructure can be version controlled, updated continuously, and rolled back like code. The meetup demonstrated setting up infrastructure on Azure using Terraform to define resources like virtual machines in code. Advanced techniques like storing state remotely and using modules were also discussed.
How to test infrastructure code: automated testing for Terraform, Kubernetes,...Yevgeniy Brikman
This talk is a step-by-step, live-coding class on how to write automated tests for infrastructure code, including the code you write for use with tools such as Terraform, Kubernetes, Docker, and Packer. Topics covered include unit tests, integration tests, end-to-end tests, test parallelism, retries, error handling, static analysis, and more.
Kubernetes is an open-source container management platform. It has a master-node architecture with control plane components like the API server on the master and node components like kubelet and kube-proxy on nodes. Kubernetes uses pods as the basic building block, which can contain one or more containers. Services provide discovery and load balancing for pods. Deployments manage pods and replicasets and provide declarative updates. Key concepts include volumes for persistent storage, namespaces for tenant isolation, labels for object tagging, and selector matching.
This document provides information about Azure DevOps and DevOps practices. It discusses how DevOps brings together people, processes, and technology to automate software delivery and provide continuous value to users. It also outlines some key DevOps technologies like continuous integration, continuous delivery, and continuous monitoring. Additionally, the document shares how Azure DevOps can help teams deliver software faster and more reliably through tools for planning, source control, building, testing, and deploying.
This document discusses infrastructure as code and the HashiCorp ecosystem. Infrastructure as code allows users to define and provision infrastructure through code rather than manual configuration. It can be used to launch, create, change, and downscale infrastructure based on configuration files. Tools like Terraform allow showing what changes will occur before applying them through files like main.tf and variables.tf. Terraform is part of the broader HashiCorp ecosystem of tools.
This document discusses Terraform, an open-source infrastructure as code tool. It begins by explaining how infrastructure can be defined and managed as code through services that have APIs. It then provides an overview of Terraform, including its core concepts of providers, resources, and data sources. The document demonstrates Terraform's declarative configuration syntax and process of planning and applying changes. It also covers features like modules, state management, data sources, and developing custom plugins.
The document discusses infrastructure as code best practices on AWS. It provides an overview of using AWS CloudFormation to define infrastructure in code. AWS CloudFormation allows infrastructure to be provisioned in an automated and repeatable way using templates that are version controlled like code. The document outlines the key components of a CloudFormation template including parameters, mappings, resources, outputs and conditionals. It also discusses using CloudFormation to bootstrap applications on EC2 instances.
***** DevOps Masters Program : https://www.edureka.co/masters-program/devops-engineer-training *****
This DevOps Docker Commands tutorial ( Docker Tutorial Blog Series: https://goo.gl/z93Ed1 ) will introduce you to the most commonly used Docker commands. The Hands-On session is performed on an Ubuntu-64 bit machine in which Docker is installed.
To learn how Docker can be used to integrate multiple DevOps tools, watch the video titled 'DevOps Tools', by clicking this link: https://goo.gl/up9iwd
Terraform is a tool for building, changing, and version controlling infrastructure safely and efficiently. It allows users to define and provision resources through code rather than manual configuration. Key concepts include variables, outputs, providers that interface with cloud platforms, reusable modules, and managing infrastructure through commands like plan, apply, and destroy. The presentation demonstrated Terraform's capabilities with a live demo and provided additional resources for learning more.
As part of this presentation we covered basics of Terraform which is Infrastructure as code. It will helps to Devops teams to start with Terraform.
This document will be helpful for the development who wants to understand infrastructure as code concepts and if they want to understand the usability of terrform
This document provides an introduction and overview of Ansible, including its main features, installation process, inventory file configuration, ad-hoc command execution, playbook usage, roles, variables, and conditions. Ansible is an automation tool that can configure systems, deploy software, and orchestrate more complex IT workloads. It uses SSH and does not require installing any agents on remote systems. Playbooks allow defining entire deployment processes as code for multi-machine orchestration.
Introduction to Jenkins and how to effectively apply Jenkins to your projects.
Jenkins Growth , Companies using Jenkins , Most downloaded and Used Plugins.
If you’re working with just a few containers, managing them isn't too complicated. But what if you have hundreds or thousands? Think about having to handle multiple upgrades for each container, keeping track of container and node state, available resources, and more. That’s where Kubernetes comes in. Kubernetes is an open source container management platform that helps you run containers at scale. This talk will cover Kubernetes components and show how to run applications on it.
Terraform is an open source tool for building, changing, and versioning infrastructure safely and efficiently. It allows users to define and provision a datacenter infrastructure using a high-level configuration language known as HashiCorp Configuration Language. Some key features of Terraform include supporting multiple cloud providers and services, being declarative and reproducible, and maintaining infrastructure as code with immutable infrastructure. It works by defining configuration files that specify what resources need to be created. The configuration is written in HCL. Terraform uses these files to create and manage infrastructure resources like VMs, network, storage, containers and more across multiple cloud platforms.
Building infrastructure as code using Terraform - DevOps KrakowAnton Babenko
This document provides an overview of a DevOps meetup on building infrastructure as code using Terraform. The agenda includes Terraform basics, frequent questions, and problems. The presenter then discusses Terraform modules, tools, and solutions. He addresses common questions like secrets handling and integration with other tools. Finally, he solicits questions from the audience on Terraform use cases and challenges.
Choosing PaaS: Cisco and Open Source Options: an overviewCisco DevNet
This document discusses container platforms and PaaS. It provides context on containers and supporting technologies like Docker. It describes how containers are limited when confined to a single host, and how schedulers can distribute containers across multiple hosts. It outlines common production tools used with containers like configuration management, monitoring, and logging. It compares PaaS and containers, noting how PaaS consumed containers before they were widely known, and how the lines between the two are blurring as container platforms provide more services. It introduces Mantl as Cisco's container stack designed to run container workloads and big data applications across clouds.
This document provides an overview and introduction to Terraform, including:
- Terraform is an open-source tool for building, changing, and versioning infrastructure safely and efficiently across multiple cloud providers and custom solutions.
- It discusses how Terraform compares to other tools like CloudFormation, Puppet, Chef, etc. and highlights some key Terraform facts like its versioning, community, and issue tracking on GitHub.
- The document provides instructions on getting started with Terraform by installing it and describes some common Terraform commands like apply, plan, and refresh.
- Finally, it briefly outlines some key Terraform features and example use cases like cloud app setup, multi
This document provides an overview and agenda for a Docker networking deep dive presentation. The presentation covers key concepts in Docker networking including libnetwork, the Container Networking Model (CNM), multi-host networking capabilities, service discovery, load balancing, and new features in Docker 1.12 like routing mesh and secured control/data planes. The agenda demonstrates Docker networking use cases like default bridge networks, user-defined bridge networks, and overlay networks. It also covers networking drivers, Docker 1.12 swarm mode networking functionality, and how concepts like routing mesh and load balancing work.
This document summarizes a meetup about infrastructure as code. It discusses the differences between treating infrastructure as "pets" versus "cattle", where pets are cared for individually and cattle are treated as disposable. When infrastructure is coded declaratively using tools like Terraform, the infrastructure can be version controlled, updated continuously, and rolled back like code. The meetup demonstrated setting up infrastructure on Azure using Terraform to define resources like virtual machines in code. Advanced techniques like storing state remotely and using modules were also discussed.
How to test infrastructure code: automated testing for Terraform, Kubernetes,...Yevgeniy Brikman
This talk is a step-by-step, live-coding class on how to write automated tests for infrastructure code, including the code you write for use with tools such as Terraform, Kubernetes, Docker, and Packer. Topics covered include unit tests, integration tests, end-to-end tests, test parallelism, retries, error handling, static analysis, and more.
Kubernetes is an open-source container management platform. It has a master-node architecture with control plane components like the API server on the master and node components like kubelet and kube-proxy on nodes. Kubernetes uses pods as the basic building block, which can contain one or more containers. Services provide discovery and load balancing for pods. Deployments manage pods and replicasets and provide declarative updates. Key concepts include volumes for persistent storage, namespaces for tenant isolation, labels for object tagging, and selector matching.
This document provides information about Azure DevOps and DevOps practices. It discusses how DevOps brings together people, processes, and technology to automate software delivery and provide continuous value to users. It also outlines some key DevOps technologies like continuous integration, continuous delivery, and continuous monitoring. Additionally, the document shares how Azure DevOps can help teams deliver software faster and more reliably through tools for planning, source control, building, testing, and deploying.
This document discusses infrastructure as code and the HashiCorp ecosystem. Infrastructure as code allows users to define and provision infrastructure through code rather than manual configuration. It can be used to launch, create, change, and downscale infrastructure based on configuration files. Tools like Terraform allow showing what changes will occur before applying them through files like main.tf and variables.tf. Terraform is part of the broader HashiCorp ecosystem of tools.
This document discusses Terraform, an open-source infrastructure as code tool. It begins by explaining how infrastructure can be defined and managed as code through services that have APIs. It then provides an overview of Terraform, including its core concepts of providers, resources, and data sources. The document demonstrates Terraform's declarative configuration syntax and process of planning and applying changes. It also covers features like modules, state management, data sources, and developing custom plugins.
The document discusses infrastructure as code best practices on AWS. It provides an overview of using AWS CloudFormation to define infrastructure in code. AWS CloudFormation allows infrastructure to be provisioned in an automated and repeatable way using templates that are version controlled like code. The document outlines the key components of a CloudFormation template including parameters, mappings, resources, outputs and conditionals. It also discusses using CloudFormation to bootstrap applications on EC2 instances.
***** DevOps Masters Program : https://www.edureka.co/masters-program/devops-engineer-training *****
This DevOps Docker Commands tutorial ( Docker Tutorial Blog Series: https://goo.gl/z93Ed1 ) will introduce you to the most commonly used Docker commands. The Hands-On session is performed on an Ubuntu-64 bit machine in which Docker is installed.
To learn how Docker can be used to integrate multiple DevOps tools, watch the video titled 'DevOps Tools', by clicking this link: https://goo.gl/up9iwd
Terraform is a tool for building, changing, and version controlling infrastructure safely and efficiently. It allows users to define and provision resources through code rather than manual configuration. Key concepts include variables, outputs, providers that interface with cloud platforms, reusable modules, and managing infrastructure through commands like plan, apply, and destroy. The presentation demonstrated Terraform's capabilities with a live demo and provided additional resources for learning more.
As part of this presentation we covered basics of Terraform which is Infrastructure as code. It will helps to Devops teams to start with Terraform.
This document will be helpful for the development who wants to understand infrastructure as code concepts and if they want to understand the usability of terrform
This document provides an introduction and overview of Ansible, including its main features, installation process, inventory file configuration, ad-hoc command execution, playbook usage, roles, variables, and conditions. Ansible is an automation tool that can configure systems, deploy software, and orchestrate more complex IT workloads. It uses SSH and does not require installing any agents on remote systems. Playbooks allow defining entire deployment processes as code for multi-machine orchestration.
Introduction to Jenkins and how to effectively apply Jenkins to your projects.
Jenkins Growth , Companies using Jenkins , Most downloaded and Used Plugins.
If you’re working with just a few containers, managing them isn't too complicated. But what if you have hundreds or thousands? Think about having to handle multiple upgrades for each container, keeping track of container and node state, available resources, and more. That’s where Kubernetes comes in. Kubernetes is an open source container management platform that helps you run containers at scale. This talk will cover Kubernetes components and show how to run applications on it.
Terraform is an open source tool for building, changing, and versioning infrastructure safely and efficiently. It allows users to define and provision a datacenter infrastructure using a high-level configuration language known as HashiCorp Configuration Language. Some key features of Terraform include supporting multiple cloud providers and services, being declarative and reproducible, and maintaining infrastructure as code with immutable infrastructure. It works by defining configuration files that specify what resources need to be created. The configuration is written in HCL. Terraform uses these files to create and manage infrastructure resources like VMs, network, storage, containers and more across multiple cloud platforms.
Building infrastructure as code using Terraform - DevOps KrakowAnton Babenko
This document provides an overview of a DevOps meetup on building infrastructure as code using Terraform. The agenda includes Terraform basics, frequent questions, and problems. The presenter then discusses Terraform modules, tools, and solutions. He addresses common questions like secrets handling and integration with other tools. Finally, he solicits questions from the audience on Terraform use cases and challenges.
Choosing PaaS: Cisco and Open Source Options: an overviewCisco DevNet
This document discusses container platforms and PaaS. It provides context on containers and supporting technologies like Docker. It describes how containers are limited when confined to a single host, and how schedulers can distribute containers across multiple hosts. It outlines common production tools used with containers like configuration management, monitoring, and logging. It compares PaaS and containers, noting how PaaS consumed containers before they were widely known, and how the lines between the two are blurring as container platforms provide more services. It introduces Mantl as Cisco's container stack designed to run container workloads and big data applications across clouds.
2017 Microservices Practitioner Virtual Summit: Ancestry's Journey towards Mi...Ambassador Labs
Adopting new development approaches such as containerization is a big change for traditional enterprise environments. Ancestry, the global leader in family history and consumer genomics, has been a big data company long before the term existed with billions of historical records and millions of family trees, much of which ran in a traditional IT environment. With a new flood of genomic data from its AncestryDNA test and the desire to continue to increase the speed of innovation, Ancestry adopted containerization and micro services using Kubernetes orchestration APIs. This session will describe Ancestry's journey to containerization and how a coherent and consistent API set such as Kubernetes can aid companies looking to make a similar transition. Paul MacKay, one of Ancestry’s Software Architects, will discuss what the company has learned during the past few years of development from both a technical and cultural change perspective.
Gluecon Monitoring Microservices and Containers: A ChallengeAdrian Cockcroft
This document discusses the challenges of monitoring microservices and containers. It provides six rules for effective monitoring: 1) spend more time on analysis than data collection, 2) reduce latency of key metrics to under 10 seconds, 3) validate measurement accuracy, 4) make monitoring more available than services monitored, 5) optimize for distributed cloud-native applications, 6) fit metrics to models to understand relationships. It also examines models for infrastructure, flow, and ownership and discusses speed, scale, failures, and testing challenges with microservices.
Docker moves very fast, with an edge channel released every month and a stable release every 3 months. Patrick will talk about how Docker introduced Docker EE and a certification program for containers and plugins with Docker CE and EE 17.03 (from March), the announcements from DockerCon (April), and the many new features planned for Docker CE 17.05 in May.
This talk will be about what's new in Docker and what's next on the roadmap
Enabling Microservices Frameworks to Solve Business ProblemsKen Owens
Opening keynote at Mesoscon 2015 with announcements on creating an ecosystem for developing solutions to business problems leveraging Mesos, Mantl.io, Mesosphere Infinity, ZoomData, and Project Calico to create Fog nodes for IoE use cases.
Beyond static configuration management discusses how containerization and distributed configuration management are disrupting traditional system engineering. Key developments include specialized container-centric operating systems like CoreOS, orchestration tools like Docker, Mesos, and Kubernetes, as well as configuration stores like etcd, Consul, and Zookeeper that enable dynamic configuration of distributed systems. The talk argues this represents an exciting transition period for development and operations.
Amazon EKS 그리고 Service Mesh
Kubernetes는 컨테이너 서비스를 도입하는 기업들에게 가장 있기있는 Orchestration 플랫폼입니다. 이 세션에서는 아마존에서 6월 정식 출시한 managed Kubenetes서비스인 EKS를 소개해드리며, 오픈소스 버전과의 차이점 및 장점 등에 대해 설명하고, 진보한 마이크로 서비스인 Service Mesh를 구현하는 Linkerd 소개 및 데모를 진행하고자 합니다.
This document contains a summary of qualifications and experience for an individual with over 3 years of experience in server administration, networking, and AWS management. It details education qualifications including an MCA and BSc, and professional experience providing support and managing AWS environments for several clients in roles such as Systems Engineer and Provisioning Engineer. Responsibilities included provisioning and maintaining systems, security, automation tasks, troubleshooting, and managing configurations.
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...GeeksLab Odessa
4.6.16 AI&BigData Lab
Upcoming events: goo.gl/I2gJ4H
Как устроить анализ данных 40 млн. человек за 5 лет так, чтобы это выглядело почти в реальном времени.
Monitoring in Motion: Monitoring Containers and Amazon ECSAmazon Web Services
Containers and other forms of dynamic infrastructure can prove challenging to monitor. How do you define normal, when your infrastructure is intentionally in motion and change from minute to minute? Join us as we discuss proven strategies for monitoring your containerized infrastructure on AWS and ECS.
MANTL Data Platform, Microservices and BigData ServicesCisco DevNet
The document discusses using Mantl, an open source platform, to deploy multiple services together in a shared cluster for better utilization and data sharing. It describes how Mesos provides resource isolation and scalability to run both complex services and microservices together. Examples are given of deploying Riak, Zoomdata, Streamsets, and other services on Mantl to take advantage of shared infrastructure and data. The goal is to maximize efficiency through a unified service platform that can run in hybrid cloud environments.
This Knolx session discusses how stateless and stateful applications can be deployed over Kubernetes using Stateful Sets. In the demo, we see how requirements for stateful services could be met and how Cassandra, a stateful service could be started over Kubernetes
This document discusses containerization and the Docker ecosystem. It begins by describing the challenges of managing different software stacks across multiple environments. It then introduces Docker as a solution that packages applications into standardized units called containers that are portable and can run anywhere. The rest of the document covers key aspects of the Docker ecosystem like orchestration tools like Kubernetes and Docker Swarm, networking solutions like Flannel and Weave, storage solutions, and security considerations. It aims to provide an overview of the container landscape and components.
This document provides an overview of managing containers at scale using Kubernetes. It begins with explaining why containers are useful for things like continuous development, deployment, and separation of concerns between development and operations. It then discusses problems that can arise when running containers without an orchestration tool, such as lack of monitoring and self-healing. Kubernetes is presented as a solution for orchestrating containers at scale, with concepts explained like pods, services, replication controllers, and namespaces. The document dives into Kubernetes design patterns and demonstrates continuous integration/deployment using Kubernetes and Jenkins. It concludes with an offered demo of Kubernetes features.
This session provides the attendee with an overview of our Amazon EC2 Container Service (Amazon ECS) and the benefits of running a managed cluster on AWS. We also discuss the benefits from a customer perspective.
Containers aren't just for stateless apps. You might have mission-critical stateful applications like MySQL, Kafka, or Cassandra databases that you want to run in containers as well. Sahil Sawhney gave an overview on how both; stateless and stateful services could be deployed over Kubernetes with ease thus reducing the service management efforts and making the scaling more manageable.
This pdf walks you through, the application of Stateful Sets and Operators and how they work internally to ensure state maintenance over the K8 cluster
Similar to Lessons learned from writing over 300,000 lines of infrastructure code (20)
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsYevgeniy Brikman
"All happy cloud deployments are alike; each unhappy cloud deployment is unhappy in its own way." — Leo Tolstoy, Site Reliability Engineer
At Gruntwork, I've had the chance to see the cloud adoption journeys of hundreds of companies, from tiny startups to Fortune 50 giants. I've seen those journeys go well. I've seen those journeys go poorly. In this talk, I discuss a few of the ways cloud adoption can go horribly wrong (massive cost overruns, endless death marches, security disasters), and more importantly, how you can get it right.
To help you get it right, we looked at the cloud journeys that were successful and extracted from them the patterns they had in common. We distilled all this experience down into something called the Gruntwork Production Framework, which defines five concrete steps you can follow to adopt the cloud at your own company—and hopefully, to end up with your very own happy cloud deployment.
A brief overview of what we do at Gruntwork. Learn what we mean by "DevOps as a Service" and how you can get your entire infrastructure, defined as code, in about a day. https://www.gruntwork.io/
Listen up, developers. You are not special. Your infrastructure is not a beautiful and unique snowflake. You have the same tech debt as everyone else. This is a talk about a better way to build and manage infrastructure: Terraform Modules. It goes over how to build infrastructure as code, package that code into reusable modules, design clean and flexible APIs for those modules, write automated tests for the modules, and combine multiple modules into an end-to-end techs tack in minutes.
You can find the video here: https://www.youtube.com/watch?v=LVgP63BkhKQ
The Truth About Startups: What I wish someone had told me about entrepreneurs...Yevgeniy Brikman
This is the talk I gave at MIT's Martin Center for Entrepreneurship. It's a talk I wish someone gave me when I was in college to help me think about the role of entrepreneurship and startups in my career.
You can find the video of the talk here: https://www.youtube.com/watch?v=Rus32iR_Ag0
This talk is a very quick intro to Docker, Terraform, and Amazon's EC2 Container Service (ECS). In just 15 minutes, you'll see how to take two apps (a Rails frontend and a Sinatra backend), package them as Docker containers, run them using Amazon ECS, and to define all of the infrastructure-as-code using Terraform.
Infrastructure as code: running microservices on AWS using Docker, Terraform,...Yevgeniy Brikman
This is a talk about managing your software and infrastructure-as-code that walks through a real-world example of deploying microservices on AWS using Docker, Terraform, and ECS.
To go faster in a car, you need not only a powerful engine, but also safety mechanisms like brakes, air bags, and seat belts. This is a talk about the safety mechanisms that allow you to build software faster. It's based on the book "Hello, Startup" (http://www.hello-startup.net/). You can find the video of the talk here: https://www.youtube.com/watch?v=4fKm6ImKml8
Every startup begins with an idea. This is a talk on how to come up with startup ideas and how to use validation to pick the ones worth working on. It's based on the book "Hello, Startup" (http://www.hello-startup.net/). You can find the video of the talk here: https://www.youtube.com/watch?v=GkmiE8d_5Pw
A guide to hiring based on my book, "Hello, Startup". Learn who to hire, where to find them, how to interview them, and how to make an offer they can't refuse.
Recording: https://www.youtube.com/watch?v=jaSmYLymc0U
Book: http://www.hello-startup.net
This is an excerpt from my talk "Startup DNA" (http://www.slideshare.net/brikis98/startup-dna) that just focuses on the "Speed Wins" concept. For more info, check out my book "Hello, Startup: A Programmer's Guide to Building Products, Technologies, and Teams" at http://www.hello-startup.net.
Node.js vs Play Framework (with Japanese subtitles)Yevgeniy Brikman
Video: http://www.nicovideo.jp/watch/1410857293
Here's the showdown you've been waiting for: Node.js vs Play Framework. Both are popular open source web frameworks that are built for developer productivity, asynchronous I/O, and the real time web. But which one is easier to learn, test, deploy, debug, and scale? Should you pick Javascript or Scala? The Google v8 engine or the JVM? NPM or Ivy? Grunt or SBT? Two frameworks enter, one framework leaves.
This version of the presentation has Japanese subtitles. For the English only version, see http://www.slideshare.net/brikis98/nodejs-vs-play-framework
Video: https://www.youtube.com/watch?v=b6yLwvNSDck
Here's the showdown you've been waiting for: Node.js vs Play Framework. Both are popular open source web frameworks that are built for developer productivity, asynchronous I/O, and the real time web. But which one is easier to learn, test, deploy, debug, and scale? Should you pick Javascript or Scala? The Google v8 engine or the JVM? NPM or Ivy? Grunt or SBT? Two frameworks enter, one framework leaves.
This is the English version of the presentation. For the version with Japanese subtitles, see http://www.slideshare.net/brikis98/nodejs-vs-play-framework-with-japanese-subtitles
This document discusses rapid prototyping techniques for quickly building products from ideas. Rapid prototyping allows developers to build products faster through instant feedback on code changes and leveraging existing open source libraries. It is suggested that dynamic languages, interactive development environments, and pushing code to the browser can provide very fast feedback. While prototypes may be thrown away, the rapid trial and error process of prototyping can lead to higher quality products and better engineers through continuously improving ideas.
Web pages can get very complex and slow. In this talk, I share how we solve some of these problems at LinkedIn by leveraging composition and streaming in the Play Framework. This was my keynote for Ping Conference 2014 ( http://www.ping-conf.com/ ): the video is on ustream ( http://www.ustream.tv/recorded/42801129 ) and the sample code is on github ( https://github.com/brikis98/ping-play ).
This document discusses asynchronous I/O in Java and Scala using the Play Framework. It describes how LinkedIn uses a service-oriented architecture with hundreds of services making requests to each other. It then covers how Play supports non-blocking I/O using asynchronous code, promises, and futures to allow parallel requests without blocking threads. Key points covered include using map and flatMap to transform promises and futures, handling errors and timeouts, and the benefits of non-blocking I/O for scalability.
Video of the presentation: http://www.youtube.com/watch?v=8z3h4Uv9YbE
At LinkedIn, we have started to use the Play Framework to build front-end and back-end services at massive scale. Play does things a little differently: it's a Java and Scala web framework, but it doesn't follow the servlet spec; it's fairly new, but it runs on top of robust technologies like Akka and Netty; it uses a thread pool, but it's built for non-blocking I/O and reactive programming; most importantly, it's high performance, but also high productivity. We've found that the Play Framework is one of the few frameworks that is able to maintain the delicate balance of performance, reliability, and developer productivity. In the Java and Scala world, nothing even comes close. In this talk, I'll share what we've learned so far, including details of rapid iteration with Java and Scala, the story behind async I/O on the JVM, support for real time web apps (comet, WebSockets), and integrating Play into a large existing codebase.
This is the story of the Hackday and[in]cubator programs at LinkedIn; this is the story of scalable, best-of-breed methods for driving innovation; this is the story of transformation: of your career, company, and the world.
Video of the live presentation: http://tcbaltics.com/video/jim-brikman
Startup DNA: the formula behind successful startups in Silicon Valley (update...Yevgeniy Brikman
[Updated May 5, 2017] "Successful startups are all alike; every unsuccessful startup is unsuccessful in its own way." These are my personal observations on a few traits that make startups successful. You can find a video of the talk at https://www.youtube.com/watch?v=z_D9oXCK2lM and the book at http://www.hello-startup.net/.
The document discusses LinkedIn's adoption of the Dust templating language in 2011. Some key points:
- LinkedIn needed a unified view layer as different teams were using different templating technologies like JSP, GSP, ERB.
- They evaluated 26 templating options and selected Dust as it best met their criteria like performance, i18n support, and being logic-less.
- Dust templates are compiled to JavaScript for client-side rendering and to Java for server-side rendering (SSR) through Google's V8 engine, allowing templates to work on both client and server.
- SSR addresses challenges like SEO, supporting clients without JavaScript, and i18n by rendering
SMS API Integration in Saudi Arabia| Best SMS API ServiceYara Milbes
Discover the benefits and implementation of SMS API integration in the UAE and Middle East. This comprehensive guide covers the importance of SMS messaging APIs, the advantages of bulk SMS APIs, and real-world case studies. Learn how CEQUENS, a leader in communication solutions, can help your business enhance customer engagement and streamline operations with innovative CPaaS, reliable SMS APIs, and omnichannel solutions, including WhatsApp Business. Perfect for businesses seeking to optimize their communication strategies in the digital age.
Microservice Teams - How the cloud changes the way we workSven Peters
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Crescat
Crescat is industry-trusted event management software, built by event professionals for event professionals. Founded in 2017, we have three key products tailored for the live event industry.
Crescat Event for concert promoters and event agencies. Crescat Venue for music venues, conference centers, wedding venues, concert halls and more. And Crescat Festival for festivals, conferences and complex events.
With a wide range of popular features such as event scheduling, shift management, volunteer and crew coordination, artist booking and much more, Crescat is designed for customisation and ease-of-use.
Over 125,000 events have been planned in Crescat and with hundreds of customers of all shapes and sizes, from boutique event agencies through to international concert promoters, Crescat is rigged for success. What's more, we highly value feedback from our users and we are constantly improving our software with updates, new features and improvements.
If you plan events, run a venue or produce festivals and you're looking for ways to make your life easier, then we have a solution for you. Try our software for free or schedule a no-obligation demo with one of our product specialists today at crescat.io
Unveiling the Advantages of Agile Software Development.pdfbrainerhub1
Learn about Agile Software Development's advantages. Simplify your workflow to spur quicker innovation. Jump right in! We have also discussed the advantages.
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeAftab Hussain
Understanding variable roles in code has been found to be helpful by students
in learning programming -- could variable roles help deep neural models in
performing coding tasks? We do an exploratory study.
- These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
Using Query Store in Azure PostgreSQL to Understand Query PerformanceGrant Fritchey
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Łukasz Chruściel
No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception.
In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed.
We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
Measures in SQL (SIGMOD 2024, Santiago, Chile)Julian Hyde
SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries.
SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL.
To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context.
A talk at SIGMOD, June 9–15, 2024, Santiago, Chile
Authors: Julian Hyde (Google) and John Fremlin (Google)
https://doi.org/10.1145/3626246.3653374
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
20. One trend I love: manage
(almost) everything as code
21. Manual provisioning
à Infrastructure as code
Manual server config
à Configuration management
Manual app config
à Configuration files
Manual builds
à Continuous integration
Manual deployment
à Continuous delivery
Manual testing
à Automated testing
Manual DBA work
à Schema migrations
Manual specs
à Automated specs (BDD)
22. The benefits of code:
1. Automation
2. Version control
3. Code review
4. Testing
5. Documentation
6. Reuse
45. Task Description Example tools
Install Install the software binaries and all dependencies.
Bash, Chef, Ansible,
Puppet
Configure
Configure the software at runtime: e.g., configure port
settings, file paths, users, leaders, followers, replication,
etc.
Bash, Chef, Ansible,
Puppet
Provision
Provision the infrastructure: e.g., EC2 Instances, load
balancers, network topology, security groups, IAM
permissions, etc.
Terraform,
CloudFormation
Deploy
Deploy the service on top of the infrastructure. Roll out
updates with no downtime: e.g., blue-green, rolling, canary
deployments.
Scripts, Orchestration
tools (ECS, K8S, Nomad)
Production-grade infrastructure checklist, part 1/4
46. Task Description Example tools
Security
Encryption in transit (TLS) and on disk, authentication,
authorization, secrets management, server hardening.
ACM, EBS Volumes,
Cognito, Vault, CiS
Monitoring
Availability metrics, business metrics, app metrics, server,
metrics, events, observability, tracing, alerting.
CloudWatch, DataDog,
New Relic, Honeycomb
Logs
Rotate logs on disk. Aggregate log data to a central
location.
CloudWatch Logs, ELK,
Sumo Logic, Papertrail
Backup and
restore
Make backups of DBs, caches, and other data on a
scheduled basis. Replicate to separate region/account.
RDS, ElastiCache, ec2-
snapper, Lambda
Production-grade infrastructure checklist, part 2/4
47. Task Description Example tools
Networking
VPCs, subnets, static and dynamic IPs, service discovery,
service mesh, firewalls, DNS, SSH access, VPN access.
EIPs, ENIs, VPCs,
NACLs, SGs, Route 53,
OpenVPN
High availability
Withstand outages of individual processes, EC2 Instances,
services, Availability Zones, and regions.
Multi AZ, multi-region,
replication, ASGs, ELBs
Scalability
Scale up and down in response to load. Scale horizontally
(more servers) and/or vertically (bigger servers).
ASGs, replication,
sharding, caching, divide
and conquer
Performance
Optimize CPU, memory, disk, network, GPU and usage.
Query tuning. Benchmarking, load testing, profiling.
Dynatrace, valgrind,
VisualVM, ab, Jmeter
Production-grade infrastructure checklist, part 3/4
48. Task Description Example tools
Cost optimization
Pick proper instance types, use spot and reserved
instances, use auto scaling, nuke unused resources
ASGs, spot instances,
reserved instances
Documentation
Document your code, architecture, and practices. Create
playbooks to respond to incidents.
READMEs, wikis, Slack
Tests
Write automated tests for your infrastructure code. Run
tests after every commit and nightly.
Terratest
Production-grade infrastructure checklist, part 4/4
49. Key takeaway: use a checklist to build
production-grade infrastructure.
53. We prefer tools that:
1. Define infrastructure as code
2. Are open source & popular
3. Support multiple providers
4. Support reuse & composition
5. Require no extra infrastructure
6. Support immutable infrastructure
55. Server Server Server Server Server
Networking, Load Balancers, Databases, Users, Permissions, etc
1. Deploy all the basic infrastructure
using Terraform
56. Server Server Server Server Server
Networking, Load Balancers, Databases, Users, Permissions, etc
VM VM VM VM VM
2. Configure the VMs using Packer
57. Server Server Server Server Server
Networking, Load Balancers, Databases, Users, Permissions, etc
VM VM VM VM VM
Docker
Cluster
3. Some of the VMs form a cluster
(e.g., ECS or Kubernetes cluster)
58. Server Server Server Server Server
Networking, Load Balancers, Databases, Users, Permissions, etc
VM VM VM VM VM
Docker
Cluster
Container Container
Container
Container
Container
Container
4. We use that Docker cluster to run
Docker containers
59. Server Server Server Server Server
Networking, Load Balancers, Databases, Users, Permissions, etc
VM VM VM VM VM
Docker
Cluster
Container Container
Container
Container
Container
Container
5. Under the hood: glue everything
together with Bash, Go, and Python
74. It’s tempting to define all of your
infrastructure code in 1 file / folder…
dev
qa
test
stage
prod
75. Downsides: runs slower; harder to understand;
harder to review (plan output unreadable); harder
to test; harder to reuse code; need admin
permissions; team concurrency limited to 1…
dev
qa
test
stage
prod
76. Also, a mistake anywhere could
break everything!
dev
qa
test
stage
prod
112. terraformOptions := &terraform.Options {
TerraformDir: "../examples/vault-with-elb",
}
defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)
validateServerIsWorking(t, terraformOptions)
Specify where the code lives
113. terraformOptions := &terraform.Options {
TerraformDir: "../examples/vault-with-elb",
}
defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)
validateServerIsWorking(t, terraformOptions)
Run terraform init and
terraform apply to deploy
114. terraformOptions := &terraform.Options {
TerraformDir: "../examples/vault-with-elb",
}
defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)
validateServerIsWorking(t, terraformOptions)
Validate the infrastructure
works as expected
115. // Get IPs of servers
aws.GetPublicIpsOfEc2Instances(t, ids, region)
// Make HTTP requests in a retry loop
http.GetWithRetry(t, url, 200, expected, retries, sleep)
// Run command over SSH
ssh.CheckSshCommand(t, host, "vault operator init")
Terratest has many tools built-in
for validation
116. terraformOptions := &terraform.Options {
TerraformDir: "../examples/vault-with-elb",
}
defer terraform.Destroy(t, terraformOptions)
terraform.InitAndApply(t, terraformOptions)
validateServerIsWorking(t, terraformOptions)
At the end of the test, run
terraform destroy to clean up
121. As you go up the pyramid, tests get
more expensive, brittle, and slower
Cost,
brittleness,
run time
e2e
Tests
Integration Tests
Unit Tests
122. How the test pyramid works
with infrastructure code:
123. Individual
modules
Unit tests for infrastructure code: test
individual sub-modules (keep them small!).
e2e
Tests
Integration Tests
Unit Tests
124. Multiple
modules
Integration tests for infrastructure code:
test multiple sub-modules together.
Individual
modules
e2e
Tests
Integration Tests
Unit Tests
125. Entire
stack
E2E tests for infrastructure code:
test entire environments (stage, prod).
Individual
modules
e2e
Tests
Integration Tests
Unit Tests
Multiple
modules
126. 30 – 120+
minutes
5 – 60
minutes
Note the test times! This is another
reason to use small modules.
1 – 20
minutes
e2e
Tests
Integration Tests
Unit Tests
127. Make sure to check out Terratest best
practices for how to speed things up!
130. Let’s put it all together:
checklist, tools, modules, tests
131. 1. Go through the checklist
Task Description Example tools
Security
Encryption in transit (TLS) and on disk, authentication,
authorization, secrets management, server hardening.
ACM, EBS Volumes,
Cognito, Vault, CiS
Monitoring
Availability metrics, business metrics, app metrics, server,
metrics, events, observability, tracing, alerting.
CloudWatch, DataDog,
New Relic, Honeycomb
Logs
Rotate logs on disk. Aggregate log data to a central
location.
CloudWatch Logs, ELK,
Sumo Logic, Papertrail
Backup and
restore
Make backups of DBs, caches, and other data on a
scheduled basis. Replicate to separate region/account.
RDS, ElastiCache, ec2-
snapper, Lambda