Introduction
For the Hashnode HarperDB hackathon, I built a Twitter thread saver. What does my Twitter saver do though? I should explain it first. Let me show you a demo:
As you can see from the above video, when a specific command is used on a Twitter thread, it gets saved to the database, and then it is rendered to the frontend site. Seems simple? You can check the frontend version here. And here are the repo links,
🕊 Inspiration and Real-world Usage
The Twitter tech community is awesome. People like Jessica Carter, Pratham, @DThompsonDev, and many other developers share their awesome content through Twitter threads. I always wanted to build something using which I can save and read the threads later. This is how I got the idea of building a thread saver. It was a long time plan to building such a thing but I couldn't get started. The Hashnode HarperDB hackathon pushed me to start building it. Till now, the app is in, I should say, the alpha version. I have a lot of plans with this app which I'll talk about in this article.
❌ Challenges
At first, when I started researching, I found that readwise.io works the same and in a much better way. Readwise worked as an inspiration to me.
Challenge 1: Choosing what I need
Now that I got an idea about what I want to build, I needed to explore the Twitter API and needed to know which API functionalities I exactly need. The Twitter stream API seemed perfect for me. Because I work more with JavaScript, JavaScript was my first choice. But I am thinking of building the whole backend with Golang in the future. I chose the twitter npm library to talk with the API.
Challenge 2: Writing functions to achieve what I need
My main target was to save a whole thread to the database, but the Twitter API doesn't send back the whole thread to which the tweets are replied to, rather it sends only the parent tweet's ID to which a tweet is replied. And also, I needed to know what is the first tweet of a thread and what a reply is. After finding the parent tweet and a child tweet, I needed to write a recursive function(if you don't know what a recursive function is, check this link 😉) that could go from a last replied thread to the top until it reaches the parent. This was the toughest part for me. I got confused a lot, but somehow I managed to write the functions that I needed.
Challenge 3: Connecting the Database
I shouldn't say connecting a database was a real challenge. HarperDB was my choice here. It is effortless to use a HarperDB database. For the server-side, I used the Harperive npm package. I could have used the API with Axios also, but I used the package here to explore a little more. You can check the backend repo here.
Challenge 4: Deploying
I am still a student and I don't have any credit card. I love deploying to Heroku, but because I need an always on server so that I can listen to the streaming data, I needed to use a VPS. I used Vultr for it. Learned a little bit of docker, but finally deployed without Docker to a Vultr linux server. It was also challenging for me to deploy to a VPS.
Challenge 5: Frontend
I prefer to work with pure JavaScript, but I choose Vue for the frontend for this project. I converted a complete HTML repo to a Vue repo. This was also not a challenge, to be honest. Though, few small places took a very long time. I used the HarperDB API with Axios here and used NoSQL queries. One problem with the HarperDB NoSQL query is, I cannot retrieve sorted data based on a specific field. Though it can be achieved in the front end, for now, it's on my to-do list. You can view the working frontend at threadsaver.xyz and the repo are here.
🛣 Future Roadmap
- The app currently lacks one of the most important features, when a thread is saved, the user should be notified either by DM or by a thread reply. Currently, I am working to solve this. This is on my top priority list now.
- Getting data sorted by time is another important thing that I need to implement
- The authentication is not yet implemented on the client-side for this Hackathon so that everyone can view the app. As the hackathon ends, I'll add the authentication service. And after adding authentication, I'll add the option to delete a specific tweet
- The second thing I need to implement is to add OAuth so that the user doesn't have to do anything manually. They just give permission to the app and they are ready to go. In this version, I also want to implement a functionality to set up custom commands.
- The UI needs more work.
- Tag adding for threads is another important feature that I need to implement.
💻 Tech Stack
🎋 Conclusion
During building this app, I learned a lot about how APIs work, more about asynchronous functions, working with Axios and VPS, and a lot more. I thank Hashnode and HarperDB for giving this push and opportunity to work on side projects.