Here we will learn Plain Vanilla JS without using NodeJS or external libraries. We will start right out of the box with a use case by converting some Google sheet data into JSON and storing it in an AWS S3 bucket, then fetching it using plain js. Most lessons start with a hello world program and not much else but here we actually have something to program right away so we can practice our arrays and loops which are key pillars in any programming language. Plus you can have an introduction into the world of Data Science here and perhaps make a career out of it like I did.
When we first study data we think of SQL but there are many apps where SQL is overkill and not needed. In a dashboard with a few metrics we can get away with a simple JSON flat file as our data source with no relationships to other data. Dashboards can use this NoSQL format and are a popular choice for a Marketing groups reporting needs.
In order to set up our environment we only really need Google Chrome and the json chrome extension for converting spreadsheet data to json. Then we need free AWS and an S3 bucket for our generic website. Only a windows notepad will do as our IDE. We will also need a local network to fetch our data because fetching data from the C drive will not work as the javascript Fetch api uses http protocol so a web server is needed for this. Before going public on free AWS we will set up a local web server first to test our data. For this we use simple Python.
set up the environment
steps to set up a local Python server and an index file
Before creating an AWS remote server we need to set up a local web server for testing..here is how to do this
First find out if Python is installed... open a command prompt which will default to your home folder c:\Users\yourname and type in python. If version info shows then it is installed and u can go to step 6 below (but be sure that an index file is saved in your home folder first, see below)
If you dont have Python installed follow these instructions:
Python Server in windows
1 go to search area and type cmd then hit command prompt, a black screen will open with a path to your home folder (usually C:\Users\yourName)
2 type in python, if installed it will show a version number
3 if not installed then the get button shows up, press this and download will install over a few minutes (or just download python using chrome)
4 once fully installed reopen the cmd prompt and type in python again
5 version info will show....here is where we start if python is already installed
6 type in python -m http.server and this starts server (keep this cmd window open)
7 be sure you saved an index file in home folder (in file explorer click c: then Users then yourName to open home folder)
7a keep cmd open while u go to localhost in step 8...closing cmd requires having to reopen and start over
8 go to chrome and type in localhost:8000 and your default index page will appear....see below for creating an index file
Python server on a Mac
on a mac open a terminal and start with step 2 above ....except we might need to try 3 different options below depending on which version of python is pre installed. Our home folder should be the folder Python is installed in and the same as the terminal folder where we start server.
try this first
- type in python -m http.server or
- type python3 -m http.server if above does not work Hit return and Python 3 will instantly start a simple HTTP server from the directory in which the command was executed..this dir should also have an index file or option 3 if others dont work
- type in python -m SimpleHTTPServer 8000 for older versions
How to create an index (home) file in our path..save it to the same folder where the web server resides. Copy this code and save it as index.html
<!DOCTYPE html>
<html lang="en-US">
<header>
</header>
<body>
<p> hi there, this is our first html page </p>
</body>
</html>
prepare some data
1 copy this data and paste it into a blank google sheet
this is our fictitious company with an orders database
order_no,order_date,product_line,dollar_amt,product1,product2,product3
12340,01-03-22,prod1,400,400,0,0
12341,01-02-22,prod2,50,0,50,0
12342,1-16-22,prod3,50,0,0,50
12343,1-17-22,prod1,100,100,0,0
12344,1-15-22,prod2,50,0,50,0
12345,2-5-22,prod1,100,100,0,0
12346,2-6-22,prod3,20,0,0,20
12347,2-7-22,prod1,100,100,0,0
12348,3-23-22,prod2,200,0,200,0
12349,3-5-22,prod3,20,0,0,20
123410,3-29-22,prod1,100,100,0,0
123411,3-25-22,prod1,100,100,0,0
123412,4-23-22,prod1,500,500,0,0
123413,4-24-22,prod2,100,0,100,0
123414,5-10-22,prod3,50,0,0,50
123415,5-15-22,prod1,500,500,0,0
123416,5-25-22,prod2,50,0,50,0
VERY IMPORTANT - after pasting data and while it is still highlighted, in google sheets, press data then split text to columns
2 get the json chrome extension
enable chrome to save as json before creating the sheet.
I found this easy shortcut that adds a JSON icon to the google sheet toolbar...this is a chrome extension
first go to this link https://chromewebstore.google.com/detail/sheets-to-json/enmkalgdnmcaljdfkojckdbhkjmffmoa
then press add to chrome, over to the far right of page
then open a blank google sheet and u will see the JSON icon as the last item in the toolbar near top of page
3 transform your data into json
paste above data into the sheet, then text to columns, then press the json icon and go to downloads to get ur json file
4 save this json file in the same folder where python and your index file reside...I saved it as orders.json
execute our program
5 fetch data from your index file...testing our server and files config
change your index.html file to contain the following code which is different from the code we will use below to access the data from a public server
<!DOCTYPE html>
<html>
<body>
<div id="myData"></div> <!--data result displays in this html div-->
<script type="text/javascript">
convert(); //run the convert function..this converts json to html and displays to the front end in the myData div
// async is needed to run await which is the newest way to return a promise..await must be used inside a function
//When you then put await in front of a function you're instructing the program to wait until that operation is complete before moving on.
async function convert() {
//let response = await fetch('https://rickd.s3.us-east-2.amazonaws.com/orders4.json'); //get data from rickd s3 bucket in aws
//replace rickd with your bucket name
let response = await fetch('orders.json'); //get data locally instead of from s3..orders.json is in same folder where python is installed
let data = await response.json(); //getting data array in json format...waiting for all the data to come in
//then iterate over javascript array for as many times as there are js objects inside the array
for (var i = 0; i < data.length; i++) {
data[i].product1 = parseInt(data[i].product1); //this converts from string to int just the val for product1
} //end for loop
//we are rendering the array and not data in table format
myData.innerHTML = JSON.stringify(data, null, "\t"); //first stringify json object then render result inside html div tag
console.log(JSON.stringify(data, null, "\t")) //this is more readable, this is the pretty print..press ctrl-shift-J to view
//console.log(JSON.stringify(data)) //prints all as one line / a string, this is the record format
} //end function
</script>
</body>
<!-- next up..we need to return this data in table format-->
</html>
6 call up localhost:8000 and view the data
7 after playing around with json on a local server we can then create a public S3 bucket in AWS
Configuring an AWS S3 Bucket as our Public Server
The simplest way to store a JSON file remotely is in AWS S3. By not creating a schema in a traditional data server we become serverless. We are out in the AWS cloud with S3 where we can connect to our bucket link from anywhere. As said previously the noSQL S3 approach has some limitations. But it also has big benefits. When working with data, human nature tends to want to only use one table similar to the ole days where we relied heavily on one excel table. This one flat file format can handle a few metrics so we don’t bombard our audience with complexity. An ideal use case for a flat file is a simple dashboard.
First we create an AWS s3 bucket then we upload the JSON file. Here is how we do it:
1 Sign up for free tier AWS, go to S3 from the AWS console and create a unique bucket name
2 make it public, by going to bucket permissions tab to turn off block public access - go to block public access section, edit, uncheck and save
3 stay in the permissions tab then edit bucket policy, erase what is there and replace with following, then save changes (remember to replace your bucket name below with the actual name)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::your bucket name/*"
}
]
}
4 then, while still in the permissions tab go to cross origin (cors), edit and replace with below and save
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"GET",
"HEAD"
],
"AllowedOrigins": [
"*"
],
"ExposeHeaders": []
}
]
5 Go to the properties tab and press static web hosting, the last item. Then click edit then enable then save. This transforms your bucket link into a public website.
6 Go back to the objects tab and press Upload, then go to where u saved ur newly created JSON file and complete this operation. Your json file will now be in s3.
7 Once the JSON file is uploaded successfully it will appear in the objects section of ur s3 bucket. Double click the JSON file and u will be in the properties tab where u see Object overview. Click on the object url provided and raw JSON data will appear. Copy and save this url link somewhere because u will need it to fetch data from Javascript. This url link provides access to the newly created generic website, a publicly accessible S3 bucket that serves as a temporary website. You can save files to this bucket and share them with others at any time.
8 accessing our json file from AWS requires different coding than used in a local server. First we need to provide the user with a link to our web page. My public link to our fictitious orders data is
https://rickd.s3.us-east-2.amazonaws.com/orders4.json Then our code should look like this which is different from our original code. For this code instead of displaying our array we add table elements so display is in table format
<html> <!-- Function to convert JSON data to HTML table using javascript fetch api
here we are adding table elements to the for loop
-->
<body>
<div id="container"></div> <!--the result appears in this div-->
<script>
convert(); //run convert function
async function convert() {
const response = await fetch('https://rickd.s3.us-east-2.amazonaws.com/orders4.json'); //fetch orders from aws S3 bucket..can just double click
//const response = await fetch('orders.json'); //fetching from local server..need to run localhost:8000/file name.html for this
const data = await response.json();
let table = document.createElement("table"); // Create the table object
//instead of traditional for loop we are using forEach loop here..a function is required and it is in arrow notation
data.forEach((item) => {
let tr = document.createElement("tr"); //the row object is created for each curly braced item in array
let vals = Object.values(item); //obj.vals method in js returns array of vals, // Get the values of the current object
vals.forEach((element) => {
let td = document.createElement("td"); //create col element
td.innerText = element; // Set the value as the text of the table cell
tr.appendChild(td); // while in each row Append each table cell val to the table row
}); //end of inner loop
table.appendChild(tr); // Append the table row to the table
}); //end of original for each loop
document.getElementById("container").appendChild(table) //append table to container
} //end of async function
</script>
</body>
<!-- next up we add some styling and headers to our table-->
</html>
next, here is the same code where we add headers and styling
<html>
<!--here we add borders to our table cells using style tags and we also add a header row. Instead of a for loop we are using a forEach loop-->
<head>
<style>
table, th, td {
border: 1px solid black;
border-collapse: collapse;
}
td, th {
padding: 10px;
}
</style>
</head>
<body>
<!-- <div id="container"></div> -->
<script>
convert(); //run convert function .. a Function to convert JSON data to HTML table
async function convert() {
const response = await fetch('https://rickd.s3.us-east-2.amazonaws.com/orders4.json');
const data = await response.json(); //getting data array in json format
// Create the table element
let table = document.createElement("table");
// Get the keys (column names) of the first object in the JSON data
let cols = Object.keys(data[0]);
// Create the header element
let thead = document.createElement("thead");
let tr = document.createElement("tr");
// Loop through the column names and create header cells
cols.forEach((item) => {
let th = document.createElement("th");
th.innerText = item; // Set the column name as the text of the header cell
tr.appendChild(th); // Append the header cell to the header row
});
thead.appendChild(tr); // Append the header row to the header
table.append(tr) // Append the header to the table
//below populates the table , above only populates header
// Loop through the JSON data and create table rows
data.forEach((item) => {
let tr = document.createElement("tr");
// Get the values of the current object in the JSON data
let vals = Object.values(item);
// Loop through the values and create table cells
vals.forEach((elem) => {
let td = document.createElement("td");
td.innerText = elem; // Set the value as the text of the table cell
tr.appendChild(td); // Append the table cell to the table row
});
table.appendChild(tr); // Append the table row to the table
});
document.body.append(table);
//container.appendChild(table) // or Append the table to the container element
}
</script>
</body>
</html>
Part 2
Arrays, Loops and Functions, some building blocks of coding, the bedrock essentials of programming
Array - the most important data structure
In javascript our data is structured as key-value pairs surrounded by curly braces and delimited by commas, an array contains multiple javascript objects with each object representing a row of data. This is called JSON format or Javascript Object Notation.
Looping / Iteration - the most important loop is the for loop but u should study up on other types of loops. A for loop iterates over an array of data and performs an operation for each pass. In our use case it just displays the data to the frontend via a fetch request.
a function
A function is a block of code, a set of instructions, that executes in response to some kind of event like pressing a button. In some cases functions are not necessary but in other cases a function is required for some code to work properly. Functions either have a name or are annonymous. They can be called or can be self executing. The dreaded arrow notation is intimidating at first but it is used widely so get used to it.
conditional logic last but not least we also have the if / else statement. if(condition) {then do this} else {do that}
in conclusion it is necessary to study up on all these topics to gain more knowledge as I am providing just a small synopsis here in this lesson. I am providing only a roadmap, a primer of sorts, and u as the programmer must learn the rest.
Happy coding !!
addendum
our plain js, nosql use case is a bar chart where u can find the code here....at https://dev.to/rickdelpo1/stacked-bar-chart-using-a-json-data-source-plain-vanilla-javascript-plain-css-and-no-chart-libraries-2j29
about the author Rick Delpo.. https://javasqlweb.org/about.html
Top comments (0)