Skip to content

Commit

Permalink
chore: move resources to public directory
Browse files Browse the repository at this point in the history
  • Loading branch information
rootCircle committed Oct 31, 2024
1 parent de6f5d0 commit b1ae8da
Show file tree
Hide file tree
Showing 48 changed files with 668 additions and 314 deletions.
37 changes: 19 additions & 18 deletions docs/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export default defineConfig({
{ text: 'Home', link: '/' },
{ text: 'Blog', link: '/archives/' },
{ text: 'Projects', link: '/project/' },
{ text: 'Start Guide', link: '/start-guide/' },
{ text: 'Start Guide', link: '/start_guide/' },
{ text: 'Notes', link: '/notes/' },
],

Expand All @@ -23,41 +23,42 @@ export default defineConfig({
text: 'Blogs',
items: [
{ text: 'MySQL Data Purging at Scale', link: '/archives/database/purging_mysql_tables_on_scale' },
{ text: 'It was not /dev/null', link: '/archives/misc/cost_of_not_initializing_variables'},
{ text: 'Linux Drama', link: '/archives/linux/linux_drama'},
{ text: 'React Init Optimization', link: '/archives/web/create-react-app-optimize'},
{ text: 'Embracing Bugs', link: '/archives/misc/buggy_code'},
{ text: 'Some Tweaks', link: '/archives/linux/tweaks'},
{ text: 'Nix', link: '/archives/linux/nix'},
{ text: 'It was not /dev/null', link: '/archives/misc/cost_of_not_initializing_variables' },
{ text: 'Linux Drama', link: '/archives/linux/linux_drama' },
{ text: 'React Init Optimization', link: '/archives/web/create-react-app-optimize' },
{ text: 'Embracing Bugs', link: '/archives/misc/buggy_code' },
{ text: 'Some Tweaks', link: '/archives/linux/tweaks' },
]
},
{
text: 'Projects',
items: [
{ text: 'cpast', link: '/project/cpast'},
{ text: 'Ideas that won\'t make you millionaire', link: '/project/ideas-that-wont-make-millionaire'}
{ text: 'cpast', link: '/project/cpast' },
{ text: 'cpast 10k special', link: '/project/cpast_10k' },
{ text: 'Ideas that won\'t make you millionaire', link: '/project/ideas_that_wont_make_millionaire' },
{ text: 'Summer Projects to pick', link: '/project/summer_projects' },
]
},
{
text: 'Start Guide',
items: [
{ text: 'Assembly 101', link: '/start-guide/assembly' }
{ text: 'Assembly 101', link: '/start_guide/assembly' }
]
},
{
text: 'Notes',
items: [
{ text: 'Distributed Systems', link: '/notes/distributed_systems'},
{ text: 'git', link: '/notes/git'},
{ text: 'Regex Notes', link: '/notes/regex'},
{ text: 'PDF Cracking', link: '/notes/pdfcrack'},
{ text: 'SpringBoot Notes', link: '/notes/springboot'},
{ text: 'HTML to React Initiative', link: '/notes/html2react'},
{ text: 'Distributed Systems', link: '/notes/distributed_systems' },
{ text: 'git', link: '/notes/git' },
{ text: 'Regex Notes', link: '/notes/regex' },
{ text: 'PDF Cracking', link: '/notes/pdfcrack' },
{ text: 'SpringBoot Notes', link: '/notes/springboot' },
{ text: 'Notes to self', link: '/archives/misc/notes_to_self' },
{ text: 'HTML to React Initiative', link: '/notes/html2react' },
{ text: 'Nix', link: '/archives/linux/nix' },
]
},
],


socialLinks: [
{ icon: 'github', link: 'https://github.com/rootCircle/blog' }
],
Expand Down
10 changes: 5 additions & 5 deletions docs/archives/database/purging_mysql_tables_on_scale.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ head:
content: website
- - meta
- property: og:image
content: https://raw.githubusercontent.com/rootcircle/blog/main/docs/archives/database/data/banner.jpg
content: https://raw.githubusercontent.com/rootcircle/blog/main/docs/public/archives/database/banner.jpg
- - meta
- name: twitter:card
content: summary_large_image
Expand All @@ -28,7 +28,7 @@ Simple-sounding problems can become a real pain when scaled; what works for ten

## Introduction

![Banner Image Placeholder](./data/banner.jpg)
![Banner Image Placeholder](/archives/database/banner.jpg)

In the fast-moving world of data management, managing large amounts of data effectively is key to keeping databases running smoothly. When data volumes grow, manual data cleaning can become difficult and prone to mistakes. To tackle this problem, many people turn to automating cleaning data in MySQL. This blog explains how to set up automated data cleaning, covers common challenges, and offers solutions for better data management.

Expand All @@ -48,7 +48,7 @@ There are three primary approaches to purge tables in MySQL clusters:

### 1. Chunk Delete

![Chunk Delete](./data/ChunkDelete.png)
![Chunk Delete](/archives/database/ChunkDelete.png)

Chunk deletion involves breaking down the delete operation into smaller, manageable chunks instead of attempting to delete large volumes of data in a single transaction. This minimizes locking and reduces the risk of replication lag.

Expand Down Expand Up @@ -92,7 +92,7 @@ This script repeats the delete operation in chunks of 1000 rows until no more ro

### 2. INSERT & TRUNCATE

![Insert & Truncate](./data/InsertTruncate.png)
![Insert & Truncate](/archives/database/InsertTruncate.png)

This approach involves copying the required data to a new table in batches and then truncating the original table to remove unwanted data. This method is useful for scenarios where a significant portion of the table needs to be purged.

Expand Down Expand Up @@ -168,7 +168,7 @@ This script transfers data, renames the original table, reinserts the new data,
### 3. Partitioning

![Partitioning Approach](./data/PartitionPurge.png)
![Partitioning Approach](/archives/database/PartitionPurge.png)

> [!NOTE] Partition & Performance
> It is so tempting to believe that partitioning will solve performance problems. But it is so often wrong.
Expand Down
2 changes: 0 additions & 2 deletions docs/archives/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,8 @@ Here are all the blogs I've written:
* [It was not /dev/null](</archives/misc/cost_of_not_initializing_variables>)
* [Embracing Bugs: How Writing Imperfect Code Helps You Grow](</archives/misc/buggy_code>)
* [Linux Drama](</archives/linux/linux_drama>)
* [Nix](</archives/linux/nix>)
* [Common Linux Tweaks](</archives/linux/tweaks>)
* [React Optimizations](</archives/web/create-react-app-optimize>)
* [Writing better Systems: Lessons I learnt](</archives/misc/notes_to_self>)

# Recommended Reads

Expand Down
6 changes: 4 additions & 2 deletions docs/archives/linux/linux_drama.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ People would rather live with a problem they cannot solve rather than accept a s
Apart from being the best gift to the mankind, Linux(sorry GNU/Linux) is highly extensible operating system. Compared to its competitors, like Windows and MacOS, Linux is created to sound more sane to the developers.

## Myths
![Bloated Window](data/bloated-os.jpg)

![Bloated Window](/archives/linux/bloated-os.jpg)

1. Linux is faster. (Truth is Windows is bloated)
2. Linux breaks things. (Kinda!)
3. Linux is secure. (You can't take everything for granted)
Expand Down Expand Up @@ -168,7 +170,7 @@ Gnirehtet: To reverse tether internet from PC to Phone<br>
:::


![Fandom Linux](data/reference-meme.jpg)
![Fandom Linux](/archives/linux/reference-meme.jpg)

## TBC

Expand Down
6 changes: 3 additions & 3 deletions docs/archives/misc/buggy_code.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ head:
content: website
- - meta
- property: og:image
content: https://raw.githubusercontent.com/rootCircle/blog/e90c4f0823b8dfd3ddfd245dd633345b6fd39b04/docs/archives/misc/data/bug_and_the_developer.png
content: https://raw.githubusercontent.com/rootCircle/blog/main/docs/public/archives/misc/bug_and_the_developer.png
- - meta
- name: twitter:card
content: summary_large_image
Expand All @@ -22,7 +22,7 @@ head:
## Introduction

![Developer sitting around staring at screen with bugs lying around](./data/bug_and_the_developer.png)
![Developer sitting around staring at screen with bugs lying around](/archives/misc/bug_and_the_developer.png)

::: info _Have no fear of perfection—you’ll never reach it._

Expand All @@ -41,7 +41,7 @@ I started my programming journey in the early 2010s. My first encounter was with

As time passed, I dabbled with HTML, QBASIC, and other languages. However, coding still didn’t feel all that exciting to me. I appreciated the idea of coding, but I couldn’t truly enjoy it.

![My life before git](./data/life_before_git.png)
![My life before git](/archives/misc/life_before_git.png)

Then came the moment when I was introduced to Python. I was in my +2 grade, and we had to create a small project for our finals. Even though it was a group project, I ended up doing most of the work! At that time, I thought GitHub was just like Google Drive for code. I had no idea what version control systems were or what "clean code" meant. And I didn't care to know. My primary goal was to make it work! I straight-up ignored things like security (SQL injections, anyone?), modularity (my project had over 6000 lines of code in a single file), and memory management issues (as long as the memory errors didn’t crash my program immediately, they weren’t a problem). And that was completely fine! This was the beginning of my journey. The project wasn’t intended to be a professional masterpiece; it was just a school assignment.

Expand Down
20 changes: 10 additions & 10 deletions docs/archives/misc/cost_of_not_initializing_variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ head:
content: website
- - meta
- property: og:image
content: https://raw.githubusercontent.com/rootCircle/blog/e90c4f0823b8dfd3ddfd245dd633345b6fd39b04/docs/archives/misc/data/bug_and_the_developer.png
content: https://raw.githubusercontent.com/rootCircle/blog/main/docs/public/archives/misc/bug_and_the_developer.png
- - meta
- name: twitter:card
content: summary_large_image
Expand All @@ -20,19 +20,19 @@ head:

> Last week, while benchmarking my Rust tool, my PC began to hang due to sudden memory spikes(sometimes > 8 GiB). The surprising twist? A simple C++ program run with nohup and /dev/null helped uncover the real culprit. Let’s dive into how this seemingly innocuous setup revealed unexpected memory issues and the lessons learned about uninitialized variables and process handling.
![Everything is fine](./data/this_is_fine.jpeg)
![Everything is fine](/archives/misc/this_is_fine.jpeg)

## Introduction

Last week, I was benchmarking one of my tools named [cpast](https://github.com/rootCircle/cpast_mono) with [hyperfine](https://github.com/sharkdp/hyperfine), and I observed something strange! With the completion of every other odd benchmark, my PC started to hang after a few seconds! The memory usage was spiking a lot ***after*** the benchmark was complete. I was pretty sure, it was not a memory leak, as I was not [leaking any memory intentionally](https://stackoverflow.com/questions/55553048/is-it-possible-to-cause-a-memory-leak-in-rust) and cpast was written in Rust as well, with all memory safety features built-in!

![Memory usage of orphan process after few seconds](./data/orphan_memusage.png)
![Memory usage of orphan process after few seconds](/archives/misc/orphan_memusage.png)

**cpast** in this context invoked multiple threads, each involving spawning a subprocess(to a g++ compiled binary with some I/O). Now, the bug with this design was that, if one thread senses something abrupt, I will explicitly call `exit()` from the program itself, sadly making spawned subprocesses in other sister threads orphan! Now, there is nothing bad with these, we can clean those as well later! But, what caught my attention was this orphan was taking so huge of memory. After all, it was a simple D.S.A. code simplistic enough I couldn't question its innocence!

There was also one more strange phenomenon, after some time, the memory usage of that orphan process would come very down and increase again in a cyclic-like fashion!

![Layout of child process under cpast](./data/cpast_child_process.png)
![Layout of child process under cpast](/archives/misc/cpast_child_process.png)

## Debugging

Expand Down Expand Up @@ -68,7 +68,7 @@ The code above is a very naive and simple code that takes a vector input from us

Now, to invoke this bug, all you need to do is compile this simple program, and start executing this under `nohup ./binary_file_name`. nohup run a command immune to hangups, with output to a non-tty! Since I hadn't given any input to the program yet, so I was pretty clueless about what was causing this sudden spike in memory usage. Logically the program should wait for the input, when given, it should do its tasks and die peacefully!

![Spawning a nohup process](./data/spawn_nohup.png)
![Spawning a nohup process](/archives/misc/spawn_nohup.png)

Out of luck, I took help from one of my college seniors and that's when we found something strange! We simplified the program a lot more and made a wild guess that the problem might be either with nohup or cin itself! But both were pretty standard software/code and for code this simple, we won't expect this to break!

Expand All @@ -78,11 +78,11 @@ Out of curiosity, we explored the man pages of nohup, it stated and I quote
Hence, the code was not getting the stdin from tty, so it should fail, right? No! Things are starting to get more weirder!

![stdin file descriptor for the orphan process](./data/stdin_file_for_nohup.png)
![stdin file descriptor for the orphan process](/archives/misc/stdin_file_for_nohup.png)

A simple `ls -al /proc/$(pgrep binary_file_name)/fd` revealed that the stdin of this process was symlinked to `/dev/null`, which in our perception was void/nothing! So, technically `cin` would not be getting the stdin file descriptor and hence it should error out? But why it didn't? We made a classic C++ joke and moved to check the same in Rust. We observed the very same behavior as C++ in Rust as well and that's the time we came to know */dev/null* was something, *something that a program can read*! Actually, */dev/null* file is an empty file and hence whenever you try reading, your OS will signal an EOF(End of File) i.e., it reads nothing and returns the flow to the program from IO. (Think of it equivalent to you having a string input, and pressing enter without any input on the console)

![stat /dev/null](./data/stat_dev_null.png)
![stat /dev/null](/archives/misc/stat_dev_null.png)

So, cin was recording an EOF, then as it couldn't read any value, and hence it wouldn't write anything to the variable as well! *(But why?)* Hence, in the program above, as all the variables were uninitialized, the value they will be storing will be some garbage value by design. Hence, the value of the **`i`** variable, even after `cin` will be some garbage and it will hence go in the while loop and hence the flow causing repeated loops and memory allocations with vector! *The same will occur for other subsequent `cin` as well!*

Expand All @@ -106,22 +106,22 @@ So, this does correctly explain the unexpectedly high increased memory usage! It

## Explanation for cpast

![cpast and the orphan process](./data/cpast_process_metadata.png)
![cpast and the orphan process](/archives/misc/cpast_process_metadata.png)

So, now if we think of *cpast*; the spawned subprocess running the compiled binary, their stdin, stdout and stderr were explicitly piped from the parent process and hence if we called exit in the main program, killing the parent process! The stdin of the spawned subprocess(now orphan) will start to pipe(point) to supposedly a dangling stdin of the parent process (which now doesn't connect to tty) and hence reads EOF!

## Question

Now, there is one questions that I still need answer about.

![scanf with stdin](./data/scanf_stdin.png)
![scanf with stdin](/archives/misc/scanf_stdin.png)

> [!TIP] The burning question
> ***If `cin`/`scanf` was reading EOF, why it didn't error out when it had to write it to an integer-type variable?***
Upon researching more, I found out that this behavior was undefined in C/C++/Rust alike and value of variable after reading an unexpected value will remain unchanged by `cin`/`scanf` alike. For more information, you can refer to this [thread on stackoverflow](https://stackoverflow.com/a/40552127).

![Same bug exists in Rust as well](./data/rust_and_this_ub.png)
![Same bug exists in Rust as well](/archives/misc/rust_and_this_ub.png)

## Conclusion

Expand Down
2 changes: 1 addition & 1 deletion docs/notes/distributed_systems.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Presto also applies dynamic filtering that can significantly improve the perform

## MapReduce

![MapReduce example](./assets/distributed_systems/mapreduce_cc_bytemonk.png)
![MapReduce example](/notes/mapreduce_cc_bytemonk.png)
source: [Video](https://www.youtube.com/watch?v=cHGaQz0E7AU)


Expand Down
3 changes: 3 additions & 0 deletions docs/notes/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,7 @@ You can find all my notes here
* [Regex](</notes/regex>)
* [Spring Boot](</notes/springboot>)
* [Distributed Systems](</notes/distributed_systems>)
* [Writing better Systems: Lessons I learnt](</notes/notes_to_self>)
* [Nix](</notes/nix>)


File renamed without changes.
File renamed without changes.
4 changes: 2 additions & 2 deletions docs/project/cpast.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ head:
<!-- </div> -->
<!---->

<video controls="controls" src="./res/cpast_walkthrough.mp4" />
<video controls="controls" src="/project/cpast_walkthrough.mp4" />

[![Crates.io](https://img.shields.io/crates/d/cpast)](https://crates.io/crates/cpast)

Expand All @@ -32,7 +32,7 @@ Are you tired of spending precious time manually debugging your code during comp

## What is cpast?

![cpast help page](./res/help_page.png)
![cpast help page](/project/help_page.png)

**cpast** is my brainchild, a tool crafted to compare outputs of two code files, regardless of the programming language used. It's the secret sauce that makes CP coding a breeze, allowing you to test solutions against random inputs. What sets cpast apart is its strong custom language, visually resembling regex, which controls input generation for testing.

Expand Down
Loading

0 comments on commit b1ae8da

Please sign in to comment.