When talking about server monitoring, most people imagine graphs and checks. Both are important to ensure that all services are up and in good shape. All the checks therefore must be clean, fast, and fully functional, that’s why we write most of our checks in Golang. However, when talking about Openstack checks, we have Gophercloud, a unique Golang library which connects all Openstack services in a single library. This article contains a quick headstart to begin using Gophercloud.
When it comes to cloud storage data, most users and companies use google drive platform. It is a good place to save your files into an online storage and therefore be able to reach your files from anywhere. Of course, even google needs to sustain its services, so some restrictions like a limited cloud space are common. When you require more practical use in your business you will reach the point of having two options.
Elasticsearch is the name of a full-text search engine in computer science, distributed for free under the Apache license. It has a RESTful interface and offers high availability, speed, and scalability. It is developed in Java and can be communicated with via the web interface. Elasticsearch is a schematic database, therefore it is not necessary to define the database structure because it is created based on embedded data. It can be included on the list of NoSQL databases.
On hot summer days when the heat is in the air, my mind starts to think about vacation and the time passing by, but business never stops and it’s nice to have all things nicely prepared before you leave the office. Especially when you can use OpenStack instrument called Heat. So, let’s take look at it a bit. Heat is a very useful orchestration tool for OpenStack users as it provides a way to automate the process of cloud components creations.
What is meant by “infrastructure as code” Infrastructure as code is a way to maintain infrastructure by automated processes and minimize human effort needed to configure anything from physical baremetals to services running on many virtual hosts. There are 2 ways to do this. The first way is to have an automating software running on every host and pulling configuration from servers which have all configuration ‘recipes’. The second way is that configuration servers push configuration to host (insecure – configuration server needs to have access to all servers).
Since our infrastructure is powered by Openstack, Cinder takes care of exposing our block devices to virtual machines. And because we value open source software, we use Ceph as the storage backend (as well as LVM in certain setups). In today’s article, I will show you the overview of Ceph architecture, pinpoint its advantages and disadvantages, and show you a demo of Ceph snapshotting to demonstrate its power and the ease of administration.
Let’s say you have just finished installing Prometheus, full of enthusiasm you want to take another step, create the structure of exporters and sort out from which exact services you want to harvest metrics. If you use it on a small scale, source code control is not your biggest concern, but when you want to collect metrics from your whole infrastructure, you definitely want to know the binaries you are running.
Golang, as a very ops/admin focused language, has a huge community and thus a lot of useful packages that can help us in the everyday development regarding monitoring, graphing, and automatization. I’m going to demonstrate a few that I use in most of my programs, either as a substitution of the default package with a similar functionality or a totally new functionality that I consider a core need of the modern ops/admin tool development.
Why Prometheus? Prometheus is an open-source system monitoring and alerting toolkit originally built at SoundCloud. Since its release in 2012, many companies and organizations have adopted Prometheus, and the project has a very active community. It is developed as an open project, independent of any company or organization.“ It is based on metrics and is designed to measure and visualise the overall health and performance of services, it is similar to tools like Graphana/Graphite, but offering a more robust and comprehensive feature set, including:
Our Infrastructure We are currently managing over 2000 virtual machines, hundreds of bare metals, tens of services, and tens of user accounts. You can imagine how difficult it was to add or change existing users (change permission, access, ssh keys, and so on). The Pains of Locally Managed Users Previously, our users were deployed only with puppet, which is great, however, searching for users in different git repositories, different branches, wasn’t the right way.
In our infrastructure we manage mainly Linux hosts, but there are also a few Windows servers that meet clients’ requirements. The best way to manage cloud infrastructure is by automation, using Puppet or Ansible for example. Unfortunately, it is only effective with a vast amount of hosts with similar features. We decided to manage all Windows hosts manually because in this case automation processes (Puppet, Ansible) would be more time consuming.
Our transition from lower Puppet version to Puppet 5 was (and still is) somewhat tricky. For us - managing more than 2000 hosts with puppet - this task is really time consuming. Every host must be switched manually to make sure no critical changes will apply. Luckily, there are some steps that can be done to simplify this task, some of which I’m going to explain. This will allow you to switch to higher version of Puppet without losing your precious data, and allow you to use other Puppet 5 features in the future.