I started this article originally to be an easy introduction to working in a terminal, using a terminal-base editor (specifically vim), using a multiplexer (tmux)... After talking to a few people about it and showing them some drafts, I realized I was still assuming too much background knowledge for a lot of people. To fix that, I split this into three parts:
- WTF do these terms mean, how your OS works, and why you care - Page you're currently on
- Working with Bash and Why I Love vim - Part 2
- Multiplexers and Looking like a 1337 h4x0r - Part 3
Collectively the name (meant to be tongue-in-cheek) is How To Be a Terminal Wizard. You aren't going to be an expert from my rantings, but hopefully you'll have a good foundation and be able to use some other resources I offer to make your development environment better!
Terminal - WTF are you talking about?
Ideally if you've found my blog you're not quite this confused, but I've been surprised at how many people who start to learn to program without any idea how the machine he or she's typing on works. You can certainly get started in most languages with a cursory knowledge, but you need to know some basics to improve beyond a certain point - you're like a musician who's learned how to play an instrument, but to join into the band (ie: the computer and others like it) you need to have some idea how notes fit together.
That metaphor is more direct than you might think. If a program is the notes, then the way your program interacts with others creates the harmony. You can do alright playing by yourself, but you'll always be limited if you don't learn how to play with others.
Great, so how's the terminal fit in? You've probably seen the command line before, typed some stuff in -- ideally you got to experience that moment of joy the first time you make words appear there. To understand what it is, and why it's called a terminal, we've got to go back several years.
Back in the early days of computing, the computers themselves were BIG. Like room-sized big. They generally weren't as powerful as the phone in your pocket either, so instead of pretty graphical interfaces, you'd communicate with them via text on the screen. Specifically, you'd go to a dumb terminal: Basically a monitor and keyboard wired to the actual computer located somewhere nearby. The computer needed certain ways to communicate to the terminal itself, mostly regarding layout. Stuff like a new line here, (eventually) change colors for this word, change the font or spacing at this point; mostly things you take for granted. You, being the user of this primitive computer, may also have wanted to change colors or fonts. To communicate with the terminal, as opposed to the computer or human, engineers created this thing called escape codes.
You're probably familiar with at least one escape code already: \n, the newline. That's where the name comes from, escaping the main communication channel to communicate to the terminal handling the display.
That's why the little command line application you launch is called a terminal. No longer a dedicated machine unto itself, the program is actually a terminal emulator, handling the passing text between your brain and the innards of your computer in more-or-less the same way.
Great, so we've handled passing messages - but how do we actually communicate? That's where your operating system comes in. Computers are just doing lots of math very, very quickly. Since humans tend to not communicate that way, we need a proper interface between how the microchips compute things and ways we can communicate. Operating systems fill that gap. Specifically they offer you some kind of shell to interact with the machine. You're using one right now: whether on OSX, KDE, Windows, whatever, you're using a graphical shell (or GUI, Graphical User Interface) to run programs and modify what the computer is doing. Inside the terminal (we already know that this will be for text-based interfaces), you'll have access to a Command Line Interface (CLI).
You can find another blog post debating the merits of various CLI shells, but unless you've intentionally changed it, odds are you're looking at Bash on Macs & Linux or DOS-like/Powershell on Windows. I will mostly be focused on Bash (obviously the best ;-)), but the purposes of all of them are the same. They let you run programs, access the filesystem, and basically use the computer.
Why are developers so crazy about working in the terminal at a command line? Believe it or not, there's no mystical or solid reason for it; mostly we like it because it's a simple, standard way to interact with every program and we don't have to move our hands off the keyboard. The bigger reason is that when you're commonly editing files, running applications from a command line (most servers & heavy applications are designed to run headless - without any display - so the easiest way is often to connect to them via a remote shell like ssh), and only popping out to check on other things, you begin to appreciate the lack of context switching. I can stay focused on what I'm doing and interact with every open file the same way, because they're all running in the same environment. We'll talk more about this phenomenon in the next part.
Notably for all programmers, a CLI itself is a programming language. The shell you have open is something called a REPL: Read Evaluate Print Loop. If you think for a second, you'll probably get why - it waits to read some input, evaluates what you type in, prints the output, then loops back to waiting for input. Most scripting languages (Javascript, Python, Perl, etc) have a similar REPL.
We'll examine using a CLI effectively in the next section, but for now you can imagine that's an extra bonus for a developer, being able to program directly into the operating system (OS).
Once you're in the CLI shell (REPL), there are now two main things you type in: Instructions to the shell, or "commands" to the OS. The former instructions are like keywords in your favorite programming language, and are often the same (for, while, do, if). The latter commands are actually programs that execute and give you desired effect/ouput (like deleting a file).
That's right - basically every action you make on your computer is triggering some little program or function to execute. So how do all these different sets of instructions operate together in harmony?
WARNING: This part can be a bit complex, and you can take entire courses on operating system design, but I'm going to try to briefly explain at a high level how the OS wrangles all that complexity to a reasonable level. Don't expect the next few paragraphs to be 100% accurate, it's more to help you have a rough mental model of what's going on behind the scenes.
I already mentioned the first thing we're going to look at, the filesystem. One of the electronic gizmos inside your computer is meant to store data. You probably know this as your hard drive. On that device is a long string of 1s and 0s (binary data) that represents every file you have saved - the OS itself, movies, pictures, games - everything. One of the things an OS does is determine how to read from and write to that arbitrary string of binary. This includes things like how to specify what kind of file it is, what the name is, and when the file ends. When you read or write a file, you communicate with the OS in a standardized format to say 'Put my data in /my/directory/file.txt' or 'Give me whatever is in /other/place/thing.mp4'. Most programming languages will have further convenience functions built in like to get text automatically converted, or to stream the data as it's received from the OS.
What do I mean "stream" the data? Don't we get it all at once? This leads into the next major component your OS usually handles - process scheduling. If you open up your process manager (the infamous CTRL+ALT+DELETE window) you'll see there's all kinds of shit running. You've got music playing, a web browser, some chat applications, that work you're going to get to, plus a bunch of other stuff that seems important... what's going on there? Basically, way down at the bottom of OS, there's this thing called a kernel. It sits right on top of the hardware and handles running every program and the communication between those physical components and the programs.
That action is called process scheduling, literally scheduling what happens when. Let's pretend there's only two things happening, you have music playing and open a big text file. Your speakers can only receive so much of the music file at a time, so the kernel is sending little pieces of the file to the speaker every few milliseconds. In between those times, it's making sure your mouse is drawn at the correct position on the screen and everything you click on interacts as expected. When you opened that file, the OS launches the application (sends some more music), tells the application what file you requested (sends more music), the application asks for that file, and the OS starts reading from the hard drive. Exactly like your speakers can only handle so much audio data at a time, you can only read so much from the hard drive at any given moment - ever taken one apart and seen the disk inside? It literally has to spin to access the entire file. So the process scheduler is reading data from the disk, giving it to the application, letting the text editor do its thing with that data, sending music to the speakers, and in between all that makes sure your mouse still moves and stuff still animates or responds to clicks.
Phew, and that's just thinking about TWO things. Warned you this can get complicated. But now that also answers why you might read a file as a stream: you receieve pieces at a time off the physical storage medium, and the OS will happily give the pieces to you as it arrives. In some instances, the kernel will buffer it and then hand the entire thing to your application.
Which leads to the last thing I want to mention, memory management. Similar to the hard drive, there's another thingamabob you know of as RAM that is for less permament, faster access storage. Your executing program is itself in RAM, as well as the file you want to read. This memory management is nearly completely abstracted away in most popular programming languages, so much so that you can be unaware of it even happening, but every variable or bit of data you access is matched to a physical spot somewhere on those chips. In the case of swap, the OS will also extend this temporary memory to a spot on the hard drive, at the cost of slower read/write access.
Ok, I hear you thinking - that's all well and good but aren't you supposed to be making me some kind of command line wizard? Yes, or at least get you started. That's what this is about, starting. As you expand in your development roles and work on more complex applications - maybe even doing some kernel hacking - you'll end up running into this level of the system more often. When we continue on in part 2, with some general tips on working in a terminal & terminal-based editor, an understanding of these low-level operations helps you understand how and why things work the way they do.
If you only get one takeaway from this introductory article, I hope it's that you should be intimately familiar with how your computer works. You're standing on the shoulders of giants to jump straight into writing your sweet web app, but once your application expands you'll likely find his shoulder isn't wide enough, and some knowledge about how he supported you will be invaluable in continuing on that path.
Less whimsically, you need to know a lot of this shit to be an effective programmer, or even a competent power user. Take some time to learn about your tools and environment. A few hours of reading can save you hours of debugging time and frustration!