[Part 2] Url Shortener System Design

This articles is documentation of what i am learning about url shortener system design from a youtube video and some research in a night and decide to make this system design become a real simple application that robust and can run very fast that i write (include to learn about) in Go and Fiber Framework.

In this second part of this series is talking about how i implemented the idea of system design to become an url shortener go apps. If you are not read Part 1 of this article, please go to this bookmark below.

[Part 1] Url Shortener System Design

Now let's go the core of this articles series.

So after i understand the logic and the system design, i plan a few nights to be spent to make this become an apps. Of course, i use this project as portfolio to learn Go and Go Fiber too 😄

So what i plan is

1st night

• understand the logic and luckily the code from video is written in Python :D.
• try to code the logic and snitching all platform that needed
• try to understand about zookeeper and go library that needed.

2nd night (the frustrated night)

• found out that zookeeper go library is not maintained any more.
• found out there is very little documentation for Zookeeper restful api.
• try to find the what is another alternative for Apache Zookeeper.
• deciding to use ETCD for Apache Zookeeper as it's the more similar platform but different principal as distributed coordination services
• the rest of the night, understand the etcd workflow and api
• try to connect with etcd via etcd go library.

3rd night

• Understood the key principal of etcd and the KVs.
• try code the rest of apps with etcd

4th night (injury time and it's my anime time day)

• Finishing the apps
• Write the blog for this journey

Please read part 1 of this series to get the system design big pictures.

Let's summary the key important of this journey

The first part that need to be understand is the logic of the encoder.

url-shortener/encode.go at master · shamir92/url-shortener

In this part, i used 66 chars (the video suggest 62 chars) but the important thing of the counter is the counterDecimal function parameter. In there, the counter decimal is modulo with 66.

So the modulo of the char will get the index of the encoderChar.

With the current logic, we can understand the pattern easily because we just need the current counter and the encodeChar list (and now it's already sorted).

To minimal the impact by try to guest what is the next pattern of short url, i suggest to scramble the chars of encodeChar to become something like this.

The second part is choosing the Zookepeer or Etcd

In the begining, i am very confuse with both of this platform. Zookeeper is using node based logic and etcd using KVs(Key Value) approach.

• etcd versus other key-value stores
• Comparison between zookeeper, etcd 3 and other distributed coordination systems

The next key point is i want to have all the platform to be very lightweight, easy to scale, and

BLAZING FAST.

After some research and see the idle memory need in docker, several articles give me some pointer to choose which one is the best eventhough i need to patch the missing part of etcd from zookeeper.

• How etcd works and 6 tips to keep in mind
• Running ZooKeeper in Production | Confluent Documentation
• Storage Memory Usage Benchmark

This is the response time result when i ran in my macbook →

Once again, this response time is get because i ran in my macbook. I will try to give a real apps response time when i up in Cloud Kubernetes at AWS for real strest test.

What i wish to be done in the next part of this series is

• Set this apps to be able run in Kubernetes (Currently cannot be done, cause my Mac cannot run Kubernetes in local, i need to use AWS EKS or other cloud kubernetes. Hope i can buy a good workstation in the Q1 2023)
• Deploy this apps to AWS to be stress test with Grafana/K6

I think that is all for Part 2. See you at Part 3 when the workstation ready or i get AWS Credits again.

Thanks all.