What I'd like to give you is a Vault environment you can get up and running with in less than 25 minutes, with 4 command lines!*
- Disclaimer: Those 4 command lines work only after installing the latest versions of Git, Virtualbox, and Vagrant.1
How does that sound? A fully operational set of 3 Vault servers, with a back-end protected by ACLs?
Have you done this in just 4 command lines, before?
If not, then, well, let's go!
The diagram shows 3 Virtual Machines, on a "host-only" network. That means that, rather than being a part of the internet, they're on the equivalent of your local wireless network confined to your laptop. You know how there are Personal Pizzas? This is a Personal Network. And you don't have to worry about any of that stuff, because VirtualBox and Vagrant will do the work for you, as we'll see.
The names of the Virtual Machines, indicated in the diagram by the white rectangles, are instance5, instance6, and instance7. Each one has both Consul and Vault installed on it. The Consuls are configured to form a cluster, which acts as a storage and High Availability "backend" for the Vault servers.
Make sense so far? Good. If not, hey, there's always the comment section.
First, let's get some prerequisites out of the way.
If you get stuck with the prerequisites, tools to install, or downloading the code, please have a look at the resources on the internet, or invest a few minutes to vent in the comments.
Especially for Vagrant, the Getting Started guide takes about 30 minutes once you have Vagrant and Virtualbox installed: https://www.vagrantup.com/intro/getting-started/index.html
If you get an error with Vault working improperly, though, make a post ASAP on the Vault Discussion Group, because they can help tell you whether it's a bug or not.
OS-Specific Prerequisites
- MacOS: OSX 10.13 or later
- Windows: Windows must have Powershell 3.0 or later. If you're on Windows 7, I recommend Windows Management Framework 4.0, because it's easier to install
Install VirtualBox and Git
The first things to get are Virtualbox and Git, since they will let you run a "Virtual Machine," and allow you to get code for this walk-through, respectively.
Install Vagrant
The first tool is Vagrant, a program that helps with all the little details of development environments. Think of it like Infrastructure as Code, but for Virtual Machines, Containers, and automating the boring parts of setting up the programs you have to run on them to do development work.
Install Tools
- Make sure you have Git installed
- Install the latest version of Vagrant (NOTE: WINDOWS 7 AND WINDOWS 8 REQUIRE POWERSHELL >= 3)
- Install the latest version of VMWare or Virtualbox
Reason for this Article
While you're waiting for all this stuff to install, here's a note about why I'm writing this.
If you search for how do I set up a hashicorp vault cluster?, you'll get some results that cover the concepts, some that discuss how to use Vault's API, and some that discuss how to enable Vault Enterprise features in general.
For instance, HashiCorp has an excellent Deployment Guide, here:
And a solid Reference Architecture, here:
But there are some pre-requisites for these. For instance, in the Deployment Guide, you would first need to know how to install a Vault Server, and know how to network containers or other infrastructure to run it on, before beginning the guide.
And the Reference Architecture is more of an overview of an example architecture, rather than getting into the nitty gritty of what kinds of commands to run that aren't directly related to Vault or Consul.
These are also, appropriately, not prescriptive or opinionated about what kind of networking stack or other infrastructure with which to set up your Vault cluster.
I wrote this guide for people who, I assume, already know a little about scripting, Git, configuring new SSH connections, installing software, and Virtual Machines, because these are hard to explain and have much better resources elsewhere.
Anyway, that's why I'm writing about an easy Vault cluster setup. Install a couple of cross-platform, Open-Source tools (Vagrant, VirtualBox, and Git), run a couple of universal commands, wait 10 minutes, and voila, your new Vault Cluster is ready to play.
Screencast Example
Here's a screencast example of the setup, if you'd like to open it on the side while running through this guide:
Download the Code for the Vault Cluster Setup
Related Vendor Documentation Link: https://help.github.com/articles/cloning-a-repository
git clone https://github.com/v6/super-duper-vault-train.git
Use this Code to Make a Vault Cluster
Related Vagrant Vendor Documentation Link: https://www.vagrantup.com/intro/index.html#why-vagrant-
-
cd super-duper-vault-train
-
vagrant up
## NOTE: You may have to wait a while forvagrant up
to complete, and there will be some "connection retry" errors for a long time before a successful connection occurs, because the VM is booting. Make sure you have the latest version, and try the Vagrant getting started guide, too
NOTE: You've made it this far. Now is a great time to find your local Philz or Pete's or Starbucks to get a well deserved
<insert favorite snack or beverage>
, because thevagrant up
process... well... it takes a while. Let Vagrant do the work while you grab a perk.
-
vagrant status
-
vagrant ssh instance5
After youssh
to that VM namedinstance5
, you'll see your command prompt change to showvagrant@instance5
. You can alsovagrant ssh
to other VMs listed in the output ofvagrant status
. - You can now use Vault or Consul from within the VM for which you ran
vagrant ssh
. For example, try the commandsconsul members
orvault status
from within any of your VMs.
Vault
Explore the Vault Cluster
ps -ef | grep vault
## Check the Vault process (run while inside a Vagrant-managed Instance)
ps -ef | grep consul
## Check the Consul process (run while inside a Vagrant-managed Instance)
vault version
## Output should be Vault v0.10.2 ('3ee0802ed08cb7f4046c2151ec4671a076b76166')
consul version
## Output should show Consul Agent version and Raft Protocol version
The Vagrant boxen have the following IP addresses:
192.168.13.35
192.168.13.36
192.168.13.37
Both Vault and Consul are running on each of them.
Vault is on port 8200.
Consul is on port 8500.
Open the Links in Tabs
http://192.168.13.35:8200 (Vault)
http://192.168.13.35:8500 (Consul)
http://192.168.13.36:8200 (Vault)
http://192.168.13.36:8500 (Consul)
http://192.168.13.37:8200 (Vault)
http://192.168.13.37:8500 (Consul)
Start Vault Data
Related Vendor Documentation Link: https://www.vaultproject.io/api/system/init.html
Start Vault.
Run this curl
command on one of the Vagrant-managed VMs, or somewhere on your computer that has curl
installed.
curl -s --request PUT -d '{"secret_shares": 3,"secret_threshold": 2}' http://192.168.13.35:8200/v1/sys/init
Unseal Vault
Related Vendor Documentation Link: https://www.vaultproject.io/api/system/unseal.html
If successful, this process will unseal the Vault at 192.168.13.35:8200
. You can use the same process for 192.168.13.36:8200
and 192.168.13.37:8200
.
- Use your unseal key to replace the value for key
abcd1430890...
, and run this on the Vagrant-managed VM.
curl --request PUT --data '{"key":"abcd12345678..."}' http://192.168.13.35:8200/v1/sys/unseal
- Run that
curl
command again. But use a different value for"key":
. Replaceefgh2541901...
with a different key than you used in the previous step, from the keys you received when running thev1/sys/init
endpoint.
curl --request PUT --data '{"key":"efgh910111213..."}' http://192.168.13.35:8200/v1/sys/unseal
Non-Vagrant
Please refer to the file PRODUCTION_INSTALLATION.md in the repository we cloned earlier.
Codified Vault Policies and Configuration
To Provision Vault via its API, please refer to the
provision_vault
folder in the repository we cloned earlier.
It has data and scripts.
The data
folder's tree corresponds to the HashiCorp Vault API
endpoints, similar to the following:
https://www.hashicorp.com/blog/codifying-vault-policies-and-configuration#layout-and-design
You can use the Codified Vault
Policies and Configuration
with your initial Root token, after
initializing and unsealing Vault,
to configure Vault quickly via its API.
The .json files inside each folder
correspond to the payloads to send to Vault
via its API, but there may also be .hcl
,
.sample
, and .sh
files for convenience's sake.
Further reading for manual setup:
https://medium.com/rigged-ops/building-a-local-hashicorp-vault-cluster-5575fe322a17
-
Slippery when wet, all rights reserved, even the contradictory ones. Blades are extremely sharp. This project may contain FOSS code. Unauthorized use or reproduction of material on this web site is strictly required and subject to congratulations. Do Not Eat. π₯½ Safety Goggles Recommended. Offer valid at participating locations, which in this case means all locations. Intended to be combined with other offers or somehow cleverly duplicated. Limit one Vagrant per visit. Cash value 1/100000000th of a Bitcoin, but don't hold me to it because who knows what that means these days. This is the fine print, why are you still reading this? Really, this is getting silly, go code.Β β©
Top comments (10)
I was looking for an article on getting Vault installed on ubunto. Big new I didn't find any on the first page so I just clicked on yours. Although this article is for MacOs and windows but it kinda helped a little.
I then continued my search and on the 5th page of google I found this blog.carbonteq.com/vault-installat... and I was only able to understand it because I had read your article so ... passive thanks... I guess.
This is designed assuming that you're on your own laptop. Because everything runs on Vagrant, and Vagrant will create VirtualBoxVMs for you, these instructions will work on any operating system on which you can install Vagrant and VirtualBox.
Thanks. Although my setup is now working but anything goes wrong next time I will get virtual machine and follow the instructions.
Thats bit more than 4 lines :) but ok .. I get it ;)
// , Technically there are only 4 commands until the cluster comes up. But you're correct that just referring to those means being a little over-optimistic, because the installation of the prerequisites Git, VirtualBox, and Vagrant isn't necessarily a matter of CLI commands.
Just teasing :)
Good post tho ..
Thanks a lot NB. It was going fine and my system crashed. This is my beginning to explore Hashi Vault and this is just awesome, the way you set this up.
I did a big mistake, forgot to copy keys & root token.
System rebooted and its in sealed state.
When I first this command, it displayed all keys & root key(This where I should have saved it)
curl -s --request PUT -d '{"secret_shares": 3,"secret_threshold": 2}' 192.168.13.35:8200/v1/sys/init
Now Iam stuck in sealed status. Please suggest me
Should I destroy and recreate it, perhaps???
// ,
And
"Every Network is a Personal Network if you try hard and believe in yourself." --Nathan Basanese
Nice write up. Link to the ASCII in your console?? π¦