When you are not able to explain it with words, you may have to add pictures. And when you even do not manage it with pictures, you can still make a video.

About this book

In the year 1970, Prof. Niklaus Wirth invented the Pascal programming language to teach his students the fundamentals of computer programming. While the initial core Pascal language was designed for teaching purposes only, it was soon expanded by commercial vendors and gained some popularity. Later, Wirth presented the language Modula-2 with improved syntax and support for modules for larger projects, and the Oberon language family with additional support for Object-Oriented Programming.

The Nim programming language can be seen in this tradition, as it is basically an easy language suited for beginners with no prior programming experience, but at the same time is not restricted in any way. Nim offers all the concepts of modern and powerful programming languages in combination with high performance and some sort of universality — Nim can be used to create programs for tiny microcontrollers as well as large desktop apps and web applications.

Most books about programming languages concentrate on the language itself and assume that the reader is familiar with the foundations of computer hardware and already has some programming experience. This is generally a valid approach, as today most people are taught this fundamental knowledge, sometimes called Computer Science (CS) in school. But still, there are people who missed this introduction in school for various reasons and decide later that they need some programming skills, maybe for a technical job. And there may exist some children that are not satisfied with the introduction to computer science taught at school. So we have decided to start this book with a short introduction to fundamental concepts — most people may skip that part, but you should be really sure that you know these foundations. This book is divided into six parts. It is possible to read the parts independently in arbitrary order, but for Nim beginners, it is recommended to read them mostly in ascending order, maybe while looking already early at some interesting sections in the second half of the book. In part II, we explain the basics of computer programming step by step in a way that should enable even children to learn independently. In this part, we may repeat some of the stuff which we already mentioned in part I. We do that by intent, as some people may skip part I, and because it is generally not a bad idea to support the learning process of the reader with some repetitions. Part III will give you an overview of Nim’s standard library, which contains many useful functions and data types, that we can use in our programs to solve common tasks like input and output operations, using the file system, or sorting of data. In part IV, we will apply what we have learned by solving some common programming tasks, like sorting, searching, or converting numbers from the internal computer format to displayable text. Part V will introduce some useful external packages, which can be easily installed with one of Nim’s package managers. Nim has already a few thousand external packages — some of them may support or replace the standard library, and others offer special or advanced functionalities. Part VI of the book will finally introduce advanced concepts like asynchronous operations, threading and parallel processing, macros and meta-programming, and last but not least, Nim’s concept implementation. Some sections, that do not integrate well into the other six parts, or that are boring or useful only for a minority of Nim users, have been moved to the Appendix and may not be part of a printed copy of the book. This includes currently a short introduction to Nim’s standard package manager, Nimble.

This book is basically a traditional textbook, a very simple one with detailed explanations, so that kids from 14 years upwards can read and understand it on their own, with no, or only minimal help from adults. The English language may still be a problem for many kids not born in a country with a good English language tradition, unfortunately. But luckily automatic translations are already supported for some languages, and perhaps we will be able to offer some translated editions later, possibly a Chinese and a German translation?

In the last decades in the area of computer programming, traditional textbooks have partly been replaced by videos and "Crash course" and "Learning by doing" books. Well, maybe a good video may indeed help you start with a new language, and a video may enable people with problems reading printed texts or with difficulties concentrating on a topic for a few minutes to learn a programming language. Unfortunately, the quality of most videos is very bad; some are made by kids just having learned the first steps of computer programming themselves. And of course, watching videos does not improve the reading and concentration issues that people have. "Crash course" and "Learning by doing" books may give you a good start, but for that, we already have a lot of textual tutorials. The concern with these kinds of books is, that they may help you with solving some common tasks, but they do not really support a deeper understanding. The idea of a "Crash course" and "Learning by doing" may not be that bad in general. But in computer science, starting with a larger example application may be an overwhelming process, as you have to learn a lot of stuff in parallel. It may work for you, but there is the danger that you forget all the details very quickly again. And these kinds of books are not very helpful when you have to look up something. The other concern with "Learning by doing" in computer science is, that learning materials may have only examples which you may not really be interested in: Of course, we can create a simple chat application, a simple Twitter clone, and do some basic web scraping using async/await. Or create a basic game or a simple GUI with one of the dozen available toolkits. But what when you are not interested in chatting and twittering, and that single selected toolkit? We think that for such a case, reading the detailed examples can be very frustrating. So we would recommend that after reading the first tutorial, and maybe a few pages of this book, you just start coding with stuff you are interested in. Perhaps together with some friends? Whenever you should need some concrete help, you should find it on the Internet, using search engines, Wikipedia, or a discussion platform of your choice. And if you have really no idea whatsoever of a project with which you can start, then potentially computer programming is just not the right profession for you.

While Nim has a JavaScript backend and so supports web-related development well, this book concentrates on native code generation using the C and C++ backends. We will discuss some peculiarities of the JavaScript backend in the second half of the book, and we may give some examples for the use of the JavaScript backend in the Appendix. If you are strongly interested in web development and the JavaScript backend, then you may also consult the book Nim in Action of Dominik Picheta, which gives some detailed examples for the development of web-based software with the Nim programming language, including a simple chat application and the skeleton of a microblogging and social networking service. And you may consult the tutorials and manuals of Nim web packages like Karax, Jester, or Basolato.

This book will not try to explain things that are already explained well elsewhere, or that should have been explained well elsewhere — at least not in this first edition, where we still have so much other essential stuff. So what we will leaf out for now is the installation of the compiler, installing and using text editors or IDEs with special Nim support, using Nim package managers like Nimble, Nimph, or others, using the foreign function interface (FFI) to create bindings to C libraries, and internal compiler details like the various memory management options, all the pragmas and similar. Also, we do not intend to fill the book up with redundant stuff, like tables listing all the Nim keywords or Nim’s primitive data types and such, as you can find all that in the compiler manual easily.

While creating graphical user interfaces (GUIs) is an important topic, we can not give many details for that due to various reasons: Nim has not yet the one and only accepted GUI library, but more than 20 attempts — from pure Nim ones like NimX or Fidget, over wrapped libraries like GTK or QML, or GUIs that try to provide a native look for various operating systems like XWidgets or NiGui, to web-based GUIs. And for each of these, at least for the more serious ones, we could write a separate GUI book. So we will give only a few minimal examples for some of them in part IV or V of the book.

Also, we will not explain game programming, as game programming is a broad area, and there are already a lot of tutorials available. Maybe in later editions of the book, we will add some of these topics, e.g. game programming, as so many people like it. But we will always have to ensure that a possible printed book version will not get more than 500 pages, so we may then leave out some stuff in the printed version.

Generally, when learning a new programming language, people start with some short tutorials before really learning the language following a book. This way is indeed a good start. So we recommend you to read the short official tutorials, parts 1 and 2, and perhaps also some other tutorials freely available online. Tutorials generally only scratch the topics, so you may not be able to understand them all really well, but this way you get already a feeling for the language. There exist also some video tutorials, for the case that you have problems reading, but in that case, this book will not be of much use to you at all. When you know already some computer science and have already experience with other languages like C++, Haskell, or Rust, then the tutorials and the Nim compiler manual may be fully sufficient for you, and you may not need this book at all.

This book is based on the Nim reference implementation of the team around Mr. A. Rumpf. While the first pages of this book were written already in the spring of 2020, it should be mostly up-to-date with the current stable version Nim v1.6. We will try to keep the book up to date with further Nim releases, like a 1.8 or maybe already a 2.0 release with planned support for incremental compilation.

The source code for the book is hosted at https://github.com/StefanSalewski/NimProgrammingBook. You may use the GitHub issue tracker to point us to mistakes or unclear explanations; we will try to fix that. Please note that we are more interested in comments on the content of the book currently, not that much in typos or grammar issues. Before the book will be officially published or a printed version will be created, we will run it through some correction software or hire a professional proofreader.

New sections and serious content changes are now (end of 2021) marked with a yellow background, and not that new stuff is still marked with a light yellow. For details, you may see the change log in the appendix.

About the Author

Dr. S. Salewski studied Physics, Mathematics, and Computer Science at Hamburg University (Germany), where he got his PhD in 2005 in the area of laser physics. He has worked in the field of fiber laser physics, electronics, and software development, using languages like Pascal, Modula-2, Oberon, C, Ruby, and Nim. Some of his software projects, including the Nim GTK GUI bindings and Nim implementations of an N-dimensional RTree and a fully dynamic, constrained Delaunay triangulation, are freely available as open-source projects at https://github.com/StefanSalewski.

Part I: Introduction

For using computers and writing computer programs, you initially do not have to know many details. It is basically like driving a car: Although a car is a powerful and complicated tool, kids generally can drive it after a 3-minute introduction. Still, good racing drivers typically need a much broader insight into the inner working of all the technical components, and finally, a lot of practice.

What is a Computer?

A computer is primarily a device that can run computer programs, by following instructions about how to manipulate data.

Nearly all the computers currently in use, from the tiny ones integrated into electronic gadgets, the well-known desktop computers (PCs), to large and powerful supercomputers filling out entire rooms, work internally with digital data only.[1] Digital data are basically integer (whole) numbers encoded in binary form, which are represented by sequences of the symbols 0 and 1. We will discuss the term digital in the next section in more detail.

The most important part of a digital computer is the CPU, the Central Processing Unit. That tiny device is built of digital electronic circuits and can perform very basic mathematical and logical operations on numbers, like adding two numbers or deciding if a number is larger or smaller than another number. Most computer CPUs can only store very few numbers internally, and forget the numbers when the power is switched off. So the CPU is typically electrically connected to a RAM module, a Random Access Memory, which can store many more numbers and allow fast access to these numbers, and to a Hard disk or SSD device, which can permanently store the numbers but does not allow such fast access. The stored numbers are most often called just data — basically, that data is nothing more than numbers, but it can be interpreted in many ways, such as pictures, sounds, and much more.

The traditional hard disk drives (HDD), which store data electromechanical on rotating magnetic disks, as well as the more modern variants, the solid-state-devices (SDD), which store data using modern semiconductor technologies, can store data persistently for longer time periods, even when no electric power supply is available. Both, SSDs and HDDs, can be optionally split into multiple partitions, e.g. one or multiple OS partitions for executable programs or pure data partitions for passive data like text files or pictures. Before use, each partition is generally formatted and a file system (FS) is created. These two steps create an internal structure on the storage device, which allows us to store and retrieve individual data blocks like programs, text files, or pictures.

Nearly all of today’s desktop computers, and even most notebooks and cellphones contain not only a single CPU, but multiple CPUs, also called "Cores", so they can run different programs in parallel, or a single program can run parts of it on different CPUs, to increase performance or reduce total execution time. The so-called supercomputers can contain thousands of CPUs. Besides CPUs, most computers have also at least one GPU, a Graphic Processing Unit, that can be used to display data on a screen or monitor, maybe for doing animations in games or for playing video. The distinction between CPU and GPU is not really sharp; usually, a CPU can also display data on screens and monitors, and GPUs can do also some data processing that CPUs can do. But GPUs are optimized for the data display task.

More visible to the ordinary computer user are the peripheral devices like keyboard, mouse, screen, and perhaps a printer. These enable human interaction with the computer, but are in no way a core component of it; the computer can run well without them. In notebooks or laptop computers or in cellphones, the peripheral devices are closely integrated with the core components. All the physical parts of a computer are also called hardware, while the programs running on that hardware are called software.

A less visible but also very important class of computers are microcontrollers and so-called embedded devices, tiny pieces with typically a hull of black plastic with some electrical contacts. The devices can contain all necessary elements, that is the CPU, some RAM, and persistent storage that can store programs and data when no electric power supply is available. These devices may be restricted in computing power and the amount of data that they can store and process, but they are contained in many consumer devices. They control your washing machine, refrigerator, television, radio, and much more. Some devices in your home may even contain multiple microcontrollers and often the microcontrollers can already communicate with each other by RF (Radio-Frequency), or access by WLAN the internet, which is sometimes called the Internet of Things (IoT).

Another class of large and very powerful digital computers is called mainframe computers or supercomputers, which are optimized to process large amounts of data very fast. The key to their gigantic computing power is that many fast CPUs work in parallel — the problem or task is split into many small parts that are solved by one CPU each, and the final result is then the combination of all the solved sub-tasks. Unfortunately, it is not always possible to split large problems into smaller sub-tasks.

Digital computers are usually driven by a clock signal that pulses at a certain frequency. The CPU can do simple operations like the addition of two integers at each pulse of the clock signal. For more complicated operations like a multiplication or a division, it may need more clock pulses.

So a rough measure for the performance of a computer is the clock rate, that is the number of clock pulses per second, divided by the number of pulses that the CPU needs to perform a basic operation, multiplied by the number of CPUs or Cores that the computer can use.

A totally different kind of computers are Quantum Computers, large, expensive high-tech devices, which use the rules of quantum mechanics to calculate many computations in parallel. Today only a few of them exist, for research at universities and some large commercial institutes. Quantum computers may at some time in the future fundamentally change computing and our whole world, but they are not the topic of this book.

Analog and Digital

Whenever we measure a quantity based on some tiny base unit, then we work in the digital area, and we measure with some granularity. Our ordinary money is digital in some way, as the cent is the smallest base unit; you will never pay a fraction of a cent for something. Time can be seen as a digital quantity as long as we accept the second as the smallest unit. Even on so-called analogue watches, the second hand will generally jump forwards in steps of a second, so you can not measure fractions of a second with that watch.

An obvious analogue property is the thermodynamic temperature and its classic measurement device is the well-known capillary thermometer consisting of a glass capillary filled with alcohol or liquid mercury. When temperature increases, the liquid in a reservoir expands more than the surrounding glass and partly fills the capillary. That filling rate is an analogue measure for the temperature.

While the hourglass works digitally (you can count the tiny sand grains), the sundial does not.

Most of the quantities in our real world seem analog, and digital quantities seem to be some sort of arbitrary approximation.

But quantum mechanics has taught us that many quantities in our world really have a granularity. Physically, quantities like energy or momentum are indeed multiplies of the tiny Planck constant. Or consider electric charge, which is always a multiple of the elementary charge unit of one electron. Whenever an electrical current is flowing through an electrically conducting wire, an ionized gas, or an electrolyte like saltwater, there are flowing multiplies of the elementary charge only, never fractions of it. And of course, light and electromagnetic radiation also have some form of granularity, which the photoelectric effect, as well as Compton scattering, proves.

An important and useful property of digital signals, and digital data, is that they map directly to integral numbers.

The simplest form of digital data is binary data, which can have only two distinct values. When you use a mechanical switch to turn the light bulb in your house on, or of, you change the binary state of the bulb. And your neighbor, when watching your house, receives binary signals.[2]

Digital computers are generally using binary electric states internally — voltage or current on or off. Such an on/off state is called a bit. We will learn more about bits and binary logic later. One bit can store obviously only two states, which we may map to the numbers 0 and 1. Larger integer numbers can be represented by a sequence of multiple bits.

The Morse code was an early application to transmit messages encoded in binary form.

A very important property of digitally encoded numbers (data) is that they can be copied and transmitted exactly without loss of precision. The reason for this is that digital numbers have a well-defined clean state, there is no noise that overlays the data and may accumulate when the data is copied multiple times. Well, that statement is not really true — under bad conditions, the noise can become so large that it changes the binary state of signals. Imagine we try to transfer some whole numbers encoded in binary form, maybe by binary states encoded as voltage level 0 Volt and 5 Volts, over an electric wire and a long distance. It is clear that the long wire can pick up some electromagnetic noise that can change the true 0 Volt data to a voltage that is closer to 5 Volts than to the true 0 Volt level, so it is received incorrectly. To catch such types of transmission errors, checksums are added to the actual data. A checksum is derived by a special mathematical formula from the original data and transferred with it. The receiver applies the same formula to the received data and compares the result with the received checksum. If it does not match, then it is clear that the data transmission is corrupted, and a resend is requested.

The opposite of digital is generally called analogue, a term that is used for data that have or seems to have no granularity. For example, we speak of an analogue voltage when the voltage can have each value in a given range and when the voltage does not "jump" but change continuously.[3] For observing analogue voltages or currents, one can use a moving coil meter, a device where the current flows through a coil in a magnetic field and the magnetic force moves the hand/pointer.

We said in the previous section that nearly all of our current computers work with digital data only. Basically, that is that they work internally with integer numbers, stored in sequences of binary bits. All input for computers must have the form of integer numbers, and all output has the form of integer numbers. Whenever we want to feed computers with some sort of analogue data, like an analogue voltage, we have to convert it into a digital approximation. For that task, special devices called analog to digital converters (ADC) exist. And in some cases we have to convert the digital output data of computers to analogue signals, like when a computer plays music: The computer output in form of digital data is then converted by a device called digital to analog converter (DAC) into an analogue voltage, that generates an analogue current through a coil in the speakers of our soundbox, and that electric current in the coil generates a magnetic field which exercises mechanical forces and moves the membrane of the speaker, resulting in oscillating motions, which generates air pressure variations that our ear can detect and that we finally hear as sound.

What is an Operating System?

Most computers, from cellphones to large supercomputers, use an operating system (OS). A well-known OS is the GNU/Linux kernel. Operating systems can be seen as the initial program, that is loaded and started when we switch the computer on and that works as some kind of supervisor:[4] it can load other programs, and it distributes resources like CPU cores or RAM between multiple running programs. It also controls user input by keyboard and mouse, displays output data on the screen — as text or graphics, controls how data is loaded and stored to nonvolatile storage media like hard-disk or SSD, manages all the network traffic, and many more tasks. An important task of the OS is to allow user programs to access all the various hardware components from different vendors in a uniform, high-level manner. An OS can be seen as an intermediate layer between user programs, like a text processor or a game, and the hardware of the computer. The OS allows user programs to work on a higher level of abstraction, so they do not need to know much about the low-level hardware details.

An important ability of most modern operating systems is to run multiple system and user programs concurrent or parallel. Concurrent execution of programs means, that the execution switches very fast between all the active programs. That way the user does not notice when programs do pause for a short time interval, and all of them seem to be running all the time, but not at full speed. True parallel execution of programs indicates, that all of them can permanently run at full speed — this is only possible when the computer has multiple CPUs or a CPU with multiple physical cores.

Computer operating systems have generally a close relation to software libraries, which are collections of data types and functions working with that data types. Libraries can be a part of the OS or can be more or less independent of the OS. Libraries are software components that provide data types and functions with a well-defined interface (API, Application Programming Interface) and behavior.

Libraries can be used as shared libraries, which are single binary files stored on the file system of a computer, often with the file extension .so or .dll, which can be accessed from different computer programs simultaneously, or as static libraries, which are part of single programs. Shared libraries have some advantages: we need only one instance on the file system of the computer, and the library is loaded only once into the computer memory (RAM), even when it is used by different apps simultaneously. This saves space, and when the library has serious errors, it is in principle possible to replace the library with a corrected version, which is then used by all the software on the computer. Shared libraries often come in numbered versions, where a higher number denotes a newer, improved, or extended library version. Sometimes some of the programs we use may need still an older library version, while other software needs already a never one. In that case, our file system has to provide multiple versions of a shared library, which can be used independently. On the other hand, statically linked libraries are directly glued with a single computer program. That makes the distribution of the program easier, as it can be shipped as a single entity, and we do not have to ensure that all the needed dynamic libraries are available on the destination computer. But if a statically linked library has serious errors, then we have to replace all the programs that are linked statically with that corrupted library.

Small microcontrollers and embedded devices often do not need to use an operating system, as they generally run only one single user program and because they usually do not have a large variety of hardware components to support.

What is a User Interface

To interact with the OS and with application programs running on the computer, we need some form of a user interface. Traditional user interfaces are text-centric and often provided directly by the OS as one single text screen filling the whole display: The user has to enter textual commands, and the computer reacts with textual messages. For entering commands and data, a keyboard is used, which layout was heavily inspired by the classical mechanical typewriter. For desktop computers, the textual user interfaces have been mostly replaced or at least supported by graphical user interfaces (GUIs) for about a half-century now, and even cellphones and other electronic gadgets now use some form of a GUI for user interaction. For large mainframe computers, the textual user interface is still common. Graphical user interfaces present the user with a set of icons or widgets, often arranged in rectangular graphical boxes called windows. These windows can be moved around, resized, and can partly or fully overlap other windows. A special type of windows is called terminal-, shell- or console-windows, which behave like the traditional full-screen textual user interfaces. Graphical user interfaces allow it to interact with the computer by simple actions like clicking on buttons or by drag or wipe gestures, performed directly on a touch-sensitive display or with a device called a mouse, which mirrors its mechanical movement on the table to a graphical cursor on the computer display, and provides a set of pushbuttons which are used to initiate a click action when the mouse pointer hovers over an icon or widget. The main advantage of graphical user interfaces is, that the user does not have to remember and type in long command sequences. A set of on-screen buttons labeled with single letters can even simulate a traditional keyboard, but when the input of longer textual data is required, the physical keyboard is still used. Graphical user interfaces are sometimes even supported by speech recognition systems, which are used to enter commands or textual messages vocally. Graphical user interfaces may be very strongly coupled with the OS but are basically still system programs executed by the OS. For the Microsoft Windows OS or the macOS, this distinction is not that obvious, as the same GUI is running permanently. For other operating systems like Linux, it is more obvious, as these operating systems are sometimes used without a GUI, and because various GUI tool kits like Gnome, KDE, and many more are available.

What is Computer Programming?

Computer programming includes the creation, testing, and optimizing of computer programs.

What is a Computer Program?

A computer program is basically a sequence of numbers, which make some sense to a computer CPU, in such a way that the CPU recognizes the numbers as so-called instructions or numeric machine code, maybe the instruction to add two numbers.

The first computers, built in the 1950s, were indeed programmed by feeding sequences of plain numbers to the device. The numbers were stored on so-called punch cards, consisting of strong paper, where the numbers were encoded by holes in the cards. The holes could be recognized by electrical contacts to feed the numbers into the CPU. As plain numbers do not match well with human thinking, soon more abstract codes were used. A very direct code, which matches numerical instructions to symbols, is the assembly language. In that language, for example, the character sequence "add A0, $8" may map directly to a sequence of numbers which instructs the CPU to add the constant integer number 8 to CPU register A0, where A0 is a storage area in the CPU where numbers can be stored. As there exist many different types of CPUs, all with their own instruction sets, there exist many different assembly instruction sets, with similar, but not identical instructions. The rules that describe how these basic instructions have to look are called the syntax of the assembly language.

The numerical machine code, or the corresponding assembly language, is the most basic instruction set for a CPU. Every instruction that a CPU can execute maps to a well-defined assembly instruction. So, each operation that a computer may be able to perform can be expressed in a sequence of assembly instructions. But complicated tasks may require millions of assembly instructions, which would take humans very long to write, and even much longer to modify, proof, and debug.[5]

Just a few years after the invention of the first computers, people recognized that they would need even more abstract instruction sets, like repeated execution, composed conditionals, or other data types than plain numbers as operands. So higher-level programming languages like Algol, Fortran, C, Pascal, or Basic were created.

What is an Algorithm?

An algorithm is a detailed sequence of more or less abstract instructions to solve a specific task or to reach a goal. Cooking recipe books and car repair instructions are examples of algorithms.

The basic math operations kids learn in school — to add, multiply or divide two numbers with paper and pencil — are algorithms too. Even starting a car follows an algorithm — when the temperature is below zero, and snow covers the vehicle, then you first have to clean the windows and lights. And when you first drive again after a long break, you would have to check the tires before you start the engine. The algorithm can be carried out by strictly following the instructions — it is not necessary to really understand how and why it works.

So an algorithm is a perfect fit for a computer, as computers are excellent at following instructions without really understanding what they are trying to accomplish.

A math algorithm to sum up the first 100 natural numbers may look like this:

use two integer variables called i and sum
assign the value 0 to both variables

while i is less than 100 do:
  increase i by one
  add value of i to sum

optionally print the final value of sum

What is a Programming Language?

Most traditional programming languages were created to map algorithms to elementary CPU instructions. Algorithms typically contain nested conditionals, repetition, math operations, recovery from errors, and maybe plausibility checks. A more complicated algorithm generally can be split into various separate logical parts, which may include reading in data at one point, multiple processing steps at another, and storing, or displaying data as plain text, graphic, or animation at yet another point. This splitting into parts is mapped to programming languages by grouping tasks into subroutines, functions, or procedures which accept a set of input parameters and can return a result.

As algorithms often work not only with numbers but also with text, it makes sense to have a form of textual data type in a programming language too. And all the data types can be grouped in various ways, for example, as sequences of multiple data of the same type, like lists of numbers or names. Or as collections of different types, like name, age, and profession of a citizen in an income tax database. For all these use cases, programming languages provide some sort of support.

Compilers and Interpreters

We already learned that the CPU in the computer can execute only simple instructions, which we call numeric machine code or assembly instructions.

To run a program written in a high-level language with many abstractions, we need some sort of converter to transform that program into the basic instructions that the CPU can execute. For the conversion process, we have basically two options: We can convert the entire program into machine code, store it on disk, and then run it on the CPU. Or we can convert it into small portions, maybe line by line, and run each portion whenever we have converted it. Tools that convert the whole program first are called compilers. Compilers process the program that we have written, include other source code like needed library modules, check the code for obvious errors and then generate and store the machine code that we then can run.

Tools that process the source code in small portions, like single statements, are called interpreters. They read in a line of source code, investigate it to check if it is a valid statement, and then feed the CPU with corresponding instructions to execute it. It is similar to picking strawberries: you can pick one and eat it at once, or you can put them all into a basket and eat them later. Both interpreters and compilers have advantages and disadvantages for special use cases. Compilers can already detect errors before the program is run, and compiled programs generally run fast, as all the instructions are already available when the programs run. The compiling step takes some time, of course, at least a few seconds, but for some languages and large programs, it may take much longer. That can make the software development process slow because as you add or change code, you have to compile it before you can execute and test your program. That may be inconvenient for unskilled programmers, as they may have to do much testing. Some use a programming style that is: change a tiny bit of the source code, then run it and see what it does. But more common practice is that you think about the problem first and then write the code, which then in most cases does nearly that of what you intended. For this style of programming, you do not have to compile and execute your code that often. Compilers have one important benefit: they can detect many bugs, mostly typing errors, already in the compile phase, and they give you a detailed error message. Interpreters have the advantage that you can modify your code and immediately execute it without delay. That is nice for learning a new language and for some fast tests, but even simple typing errors can only be detected when they are encountered while running the program. If your test does not try to run a faulty statement, there will be no error, but it may occur later. Generally, interpreted program execution is much slower than running compiled executables, as the interpreter has to continually process the source code in real-time as it’s being run, while the compiler does it only once before the program is run. At the end of this section, a few additional notes:

Compilers are sometimes supported by so-called linkers. In that case, the compiler converts the source code, which can be stored in multiple text files, each in a sequence of machine code instructions, and finally the linker joins all these machine code files to the final executable. Some compilers do not need the linking step or call the linker automatically. And some interpreters convert the textual source code in one very fast, initial pre-processing step ("on the fly") to a so-called byte code, that can then be interpreted faster. The languages Ruby and Python do that. Some languages, like Java, can compile and optimize the source code while the program is running. For that process, a so-called virtual machine is used, which builds an intermediate layer between the hardware and the user program.

Types of Programming Languages

There are many different styles that software can be written. A programming paradigm is a fundamental style of writing software, and each programming language supports a different set of paradigms. You’re probably already familiar with one, or more of them, and at the very least, you know what object-oriented programming (OOP) is because it’s taught as part of many introductory computer science courses.

We already mentioned the assembly languages, which provide only the basic operations that the CPU can perform. Assembly languages provide no abstractions, so maybe we should not even call them programming languages at all. Then there are low-level languages like Fortran or C, with some basic abstractions, which still work close to the hardware and which are mostly designed for high performance and low resource consumption (RAM), but not to detect and prevent programming errors or to make life easy for programmers. These languages already support some higher-order data types, like floating-point numbers or text (strings), homogeneous, fixed-size containers (called arrays in C), or heterogeneous fixed-size containers (called structs in C).

A different approach is taken by languages like Python or Ruby, which try to make writing code easier by offering many high-level abstractions and which have better protection against errors but are not as efficient. These languages also support dynamic containers, which can grow and shrink, or advanced data structures like hash tables (maps) or support textual pattern matching by regular expressions (regex).

Another way to differentiate programming languages is if they are statically or dynamically typed. Ruby, Python, and JavaScript are all examples of dynamically typed languages, that is, they use variables that can store any data type, so the variable’s type of data that it accepts can therefore dynamically change during program execution. That seems comfortable for the user, and sometimes it is, especially for short programs, which may be written for one-time use only and are sometimes called scripts. But dynamic typing makes the discovery of logical errors harder — an illegal addition of a number to a letter may be detected only at run-time. And dynamically typed languages generally waste a lot of memory and their performance is not that great. It is as if we would own a set of large, equally sized moving boxes, and we would store all of our goods in it, each piece in one box.

For statically typed languages, each variable has a well-defined data type like integer number, real number, a single letter, a text element, and many more. The data type is either assigned by the author of the program with a type declaration, or is detected by the compiler itself when processing the program source code, called type inference, and the variable’s type does never change. In this way, the compiler can check for logical errors early in the compile process, and the compiler can reserve memory blocks exactly customized to the variables that we want to store, so total memory consumption and performance can be optimized. Referring again to our boxes example, statically typing is like using customized boxes for all your goods.

All these types of programming languages are often called imperative programming languages, as the program describes detailed what to do. There are other types of programming languages too, for example, languages like Prolog, which try to give only a set of rules and then let the computer try to solve a problem with these rules. And of course, there are the new concepts of artificial intelligence (AI) and machine learning (ML), which are less based on algorithms and more on neural nets, which are trained with a lot of data until they can provide the desired results. Nim, the computer language this book is about, is an imperative language, so we will focus on the imperative programming style in this book. But of course, Nim can be used to create AI applications.

Further still, we can differentiate between languages like C, C++, Rust, Nim, and many more that compile to native executables and can run directly on the hardware of the computer, contrasted with languages like Java, Scala, Julia, and some more, that use a large Virtual Machine (VM) as an intermediate layer between the program and the hardware, and interpreted languages like Ruby and Python. Languages using a virtual machine generally need some startup time when a program is invoked, as the VM must be loaded and initialized, and interpreted languages are generally not very fast.[6] The distinction between languages that compile to native executables, and those that are executed on a virtual machine, is not really sharp. For example, Kotlin and Julia were executed on a virtual machine initially but now can compile the source code to native executables.

An important class of programming languages is the group of so-called Object-Oriented-Programming (OOP) languages, which use inheritance and dynamic dispatch and become popular in the 1990s. For some time, it was assumed that Object-Oriented-Programming was the ultimate solution to manage and structure really large programs. Java was the most prominent example of the OOP languages. Java forces the programmer to use OOP design, and languages like C++, Python, or Ruby strongly push the programmer to use the OOP design. The practice has shown that OOP design is not the ultimate solution for all computing problems, and OOP design may prevent optimal performance. So newer languages, like Go, Rust, and Nim, support some form of OOP programming but use it only as one paradigm among many others.

Another popular and important class of programming languages is JavaScript and its more modern cousins like TypeScript, Kotlin or Dart, and others. JavaScript was designed to run in web browsers to support interactive web pages and programs and games running in the browser. In this way, the program became nearly independent of the native operating system of the computer. Note that unlike the name may indicate, JavaScript is not closely related to the Java language. Nim can compile to a JavaScript backend, so it supports web development well.

Table 1. Overview of popular programming languages:[7]
Language Paradigm Typing discipline Syntax Execution Memory Management Generics Macros, Meta-programming Modules

C

Imperative (procedural), structured

Static, weak

Braces, semicolons

Native

Manual

No

Text preprocessor

No

Nim

Imperative (procedural), structured, functional, object-oriented

Static, strong, inferred

Python-like

Native, web browser (JavaScript)

GC, refcount, destructors

Yes

AST based, hygenic

Yes

Sometimes source code written in one programming language is converted into another one. A prominent target for such conversions is JavaScript, as JavaScript enables the execution of programs in web browsers. Another important target language is C or C++. Creating intermediate C code, which is then compiled by a C compiler to native executables, has some advantages compared to direct compilation to native executables: C compilers exist for nearly all computer systems including microcontrollers and embedded systems, so the use of a language is not restricted to systems for which a native compiler backend is provided. And C as intermediate code simplifies the use of system libraries, which typically provide a C-compatible interface. Due to decades of development, C compilers generally can do better code optimizations than young languages may manage to do. Some people fear that intermediate C code carries the problems of the C language, like verbosity, confusing and error-prone code, or undefined behavior, to the source languages. But these well-known concerns of C occur only when humans write C code directly, in the same way as when humans write assembly code directly. Automatic conversions are well-defined and well tested, which means they are free of errors to the same degree as direct machine code generation would be. But indeed there are some small drawbacks when C or C++ is used as a backend of a programming language: C does not always allow direct access to all CPU instructions, which may make it difficult to generate optimal code for some special constructs like exceptions. And C uses wrap-around arithmetic for unsigned integer types, which may not be what modern languages desire. The current Nim implementation provides a JavaScript and a C and C++ backend. While the JavaScript backend is a design decision to enable web development, the C and C++ backends are a more pragmatic decision and may be later replaced or at least supported by direct native code generation or use of the popular LLVM backend. [8] When computer languages are converted from one language to another, then sometimes the term transpiler is used to differentiate the translation process from a direct compilation to a binary executable. When program code is converted between very similar languages with nearly the same level of abstractions, then the term transpiler may be justified. But Nim is very different from C and has a higher abstraction level, and the Nim compiler performs many advanced optimizations. So it should be not called a transpiler, even when compiling to JavaScript or to the C++ backend.

Why Nim?

In this section, we are using a lot of new Computer Science (CS) expressions but do not explain them. That is intentional — when you already know them, you may get a better feeling of what Nim is, and when you do not know them, you will at least learn that we can describe Nim with fancy-sounding terms.

Three well-known traditional programming languages are C, Java, and Python. C is basically a simple, close-to-the-hardware language created in 1972, for which compilers can generate fast, highly optimized native machine code, but it has cryptic syntax, some strange semantics, and is missing higher concepts of modern languages. Java, created in 1995, forces you strongly to the object-orientated style of programming (OOP) and runs on a virtual machine, which makes it unsuitable for embedded systems and microcontrollers. Python, created in 1991, is generally interpreted instead of compiled, which makes program execution not very fast, and it does not really allow writing low-level code which operates close to the hardware. As many libraries of the Python language are written in highly optimized C, Python can appear really fast if a standard task, like sorting data, processing CSV or JSON files, or website crawling is performed. So Python is not a bad solution when we use it mostly for calling library functions, but it reveals its low performance when we have to write some actual Python code in order to solve a problem. Of course, there are many more programming languages, each with its own advantages and disadvantages — with some optimized for special use cases.

Nim is a state-of-the-art programming language well suited for systems and application programming. Its clean Python-like syntax makes programming easy and fun for beginners, without applying any restrictions to experienced systems programmers. Nim combines successful concepts from mature languages like Python, Ada, and Modula with a few established features of the latest research. It offers high performance with type and memory safety while keeping the source code short and readable. The compiler itself and the generated executables support all major platforms including Windows, Linux, BSD, and Mac OS X. Cross-compiling to Android and other mobile and embedded devices and microcontrollers is possible, and the JavaScript backend allows creating web apps and to run programs in web browsers. The custom package managers, Nimble or Nimph, makes use and redistribution of programs and libraries easy and secure. Nim supports various "backends" to generate the final code. The C, C++ and LLVM-based backends allow easy OS library calls without additional glue code, while the JavaScript backend generates high-quality code for web applications. The integrated "Read/Eval/Print Loop" (REPL), "Hot code reloading", incremental compilation (expected for version 1.8), and support of various development environments including debugging and language server protocols make working with Nim productive and enjoyable.

Some Facts About Nim

  • Nim is a multi-paradigm programming language. Unlike some popular programming languages, Nim doesn’t focus on the OOP paradigm. It’s mainly an imperative and procedural programming language, with varying support for OOP, data-orientated, functional, declarative, concurrent, and other programming styles. Nim supports common OOP features, including inheritance, polymorphism, and dynamic dispatch.

  • The generated executables are dependence free and small: a simple chess program with a plain GTK-based graphical user interface is only 100 kB in size, and the size of the Nim compiler executable itself is about 6.5 MB. It is possible to shrink the executable size of "Hello World" programs to about 10 kB for use on tiny microcontrollers.

  • Nim is fast. Generally, performance is very close to other high-performance languages such as C or C++. There are some exceptions still: other languages may have libraries or applications that have been tuned for performance for many years, while similar Nim applications are so far less tuned for performance, or maybe are more written with a priority of short and clean code or run-time safety.

  • Clean Python-like syntax with significant white-space, no need for block delimiters like {} or begin/end keywords, and no need for statement delimiters like ;

  • Safety: Nim programs are type- and memory-safe — memory corruption is prevented by the compiler as long as unsafe low-level constructs like casts, pointers, and the addr operator or the {.union.} pragma are not used.

  • Fast compiler. The Nim compiler can compile itself and other medium-size packages in less than 10 seconds, and the upcoming incremental compilation will increase that speed further.

  • Nim is statically typed: each object and each variable has a well-defined type, which catches most programming errors already at compile-time, prevents run-time errors, and ensures the highest performance. At the same time, the statically typing makes it easier to understand and maintain larger codebases.

  • Nim supports various memory management strategies, including manual allocations for critical low-level tasks as well as various garbage collectors including a destructor-based, fully deterministic memory manager.

  • Nim produces native, highly optimized executables and can also generate JavaScript output for web applications.

  • Nim has a clean module concept, which helps to structure large projects.

  • Nim has a well-designed standard library that supports many basic programming tasks. The full source code of the library is included and can be viewed easily from within the HTML-based API documentation.

  • Library modules like the os module provide OS-independent abstractions, which allow for the compilation and running of the same program on different operating systems without modifications.

  • The Nim standard library is supported by more than 1000 external packages for a broad range of use cases. External packages can be installed easily with Nim’s package managers.

  • Asynchronous operation, threading and parallel processing is supported.

  • Nim supports all popular operating systems including Linux, Windows, macOS, and Android.

  • Usage of external libraries written in C is easy and occurs directly without any glue code, and Nim can even work together with code written in other languages, for example, there are some Nim <-> Python interfaces available.

  • Many popular editors have support for Nim syntax highlighting and other IDE functionality like on-the-fly checking for errors and displaying detailed information about imported functions and data types.

  • In the last few years, Nim has reached some important milestones: Version 1.0 with some stability promises was released, and with the ARC and ORC memory management strategies and full destructor support fully deterministic memory management comparable to memory management in C++ or Rust is available. So problems of conventional garbage collectors like delayed memory deallocation or longer pausing of programs due to the GC process are gone. And some larger companies have started using Nim in production, the most influential may be currently the Status Corp. with their Ethereum client development.

Nim supports many programming styles

We mentioned already that Nim is a multi-paradigm programming language, that supports various programming styles. While we may regard Nim in the first line as an imperative, procedural programming language, it supports the popular functional and object-orientated programming styles well.

In classical OOP programming languages, we have the concept of classes with attributes, and methods that are very closely bound to the classes, as in Python:

class User:
  def say(self):
    print("It does not work!")

user = User()
user.say()

In this Python snippet, we declare a class User, with a custom method named say() bound to this class. Then we create an instance variable of this class and call its say() method.

This tight binding of methods to classes is not very flexible, e.g. extending the set of methods of a class may be difficult or impossible. Another problem with such a class concept is, that it is not always clear to which class a method belongs when more than just one single class is involved: Imagine that we need a method that appends a single character to a text string. Is that method a member of the character class, or a member of the text string class?

Nim avoids such a strict class concept, while its generalized method call syntax allows us to use a class like syntax for all of our data types: e.g. to get the length of a string variable, we can write len(myString) in classical procedural notation, or we can use the method call syntax myString.len() or just myString.len. The compiler regards all these notations as equivalent, so we have the method syntax available without the restrictions of the class concept. The method call syntax can be used in Nim for all data types, even for plain numbers — so the notation abs(myNum) is fully equivalent with myNum.abs.

The Python code from above may look in Nim like

type User = object

proc say(self: User) =
  echo ("It does not work!")

let user = User()
user.say()

Instead of classes, we use object types in Nim, and we define procedures and methods that can work on objects or other data types.

As an example of the functional programming style in Nim, we may look at some code fragment from a real-world app that has to generate a string from four numbers, separated by commas. Using the mapIt() procedure imported from the sequtils module and the fmt() macro from the strformat module, we may write that in functional programming style in this way:

from strutils import join
from sequtils import mapIt
from strformat import fmt
const DefaultWorldRange = [0.0, 0, 800, 600]
let str = DefaultWorldRange.mapIt(fmt("{it:g}")).join(", ")
echo str # "0, 0, 800, 600"

In the imperative, procedural style, we would write it like

var str: string
for i, x in pairs(DefaultWorldRange):
  str.add(fmt("{x:g}"))
  if i < DefaultWorldRange.high:
    str.add(", ")

Nim is Efficient

Nim is a compiled and statically-typed language. While for interpreted, dynamically-typed languages like Python we have to run every statement to check even for trivial errors, the Nim compiler checks for most errors during the compile process. The static typing together with the well-designed Nim type system allows the compiler to catch most errors already in the compile phase, like the undefined addition of a number and a letter, and to report the errors in the terminal window or directly in the editor or IDE. When no errors are found or all errors have been fixed, then the compiler generates highly optimized dependency-free executables. And this compilation process is generally really fast, for example, the compiler compiles itself in maybe 10 to 30 seconds on a typical modern PC.[9]

Modern concepts like zero-overhead iterators, compile-time evaluation of user-defined functions, and cross-module inlining in combination with the preference for value-based, stack-located data types lead to extremely efficient code. Multi-threading, asynchronous input/output operations (async IO), parallel processing, and SIMD instructions including GPU execution are supported. Various memory management strategies exist: selectable and tuneable high-performance Garbage Collectors (GC), including a new fully deterministic destructor-based GC, are supported for automatic memory management. These can be disabled for manual memory management. This makes Nim a good choice for application development and close-to-the-hardware system programming at the same time. The unrestricted hardware access, small executables, and optional GC will make Nim a perfect solution for embedded systems, hardware drivers, and operating system development.

Nim is Expressive and Elegant

Nim offers a modern type system with templates, generics, and type inference. Built-in advanced data types like dynamic containers, sets, and strings with full UTF support are completed by a large collection of library types like hash tables and regular expressions. While the traditional Object-Oriented-Programming programming style with inheritance and dynamic dispatch is supported, Nim does not enforce this programming paradigm and offers modern concepts, like procedural and functional programming. The optional method call syntax allows to use all data types and functions in an OOP like fashion, e.g. instead of len(myStr) we can also use the OOP style myStr.len.[10] The powerful AST-based hygienic macro system offers nearly unlimited possibilities for the advanced programmer. This macro and meta-programming system allows compiler-guided code generation at compile-time, so the Nim core language can be kept small and compact, while many advanced features are enabled by user-defined macros. For example, the support of asynchronous IO operations has been created with these forms of meta-programming, as well as many Domain Specific Language (DSL) extensions.

Nim is Open and Free

The Nim compiler and all modules of the standard library are implemented in Nim. All source code is available under the less restricted MIT license.

Nim has a community

The Nim forum is hosted at:

and the software running the forum is coded in Nim.

Real-time chat is supported by IRC, Gitter, Discord, Telegram and others.

Nim is also supported by Reddit.com and Stackoverflow.com:

Nim is evolving

Started more than 12 years ago as a small community project of some bright CS students led by Mr. A. Rumpf, it is now considered as one of the most interesting and promising programming languages, supported by countless individuals and leading companies of the computer industry, for instance, it’s actively used in the areas of application, game, web, and cryptocurrency development. Nim has made a large amount of progress in the last few years: it reached version Nim v1.6 with some stability guarantees and a new deterministic memory management system was introduced, which will improve support of parallel processing and the use of Nim in the area of embedded systems development.

Nim was created by Mr. A. Rumpf in 2008, supported by a few volunteers. Finally, in 2018 Nim got some significant monetary support from Status Corp. and in 2019 the stable Nim version 1.0 was released. But still, Nim is developed by a small core team and some volunteers, while some other languages like Java, C#, Go, or Rust are supported by large companies, or like C and C++ have a very long history and well-trained users. And finally, there are many competing languages, some with a longer history, and some possibly better suited for special purposes, like JavaScript, Dart or Kotlin for web development, Julia or R for numeric applications, or Zig, C and Assembly for the tiny 8-bit microcontrollers with a small amount of RAM.

Nim is already supported by more than 1000 external packages which cover many application areas, but that number is still small compared to really popular languages like Python, Java, or JavaScript. And some Nim packages can currently not really compare with the libraries of other languages, which have been optimized for years by hundreds or thousands of full-time developers.

Indeed, the future of Nim is not really secure. Core developers may vanish, financial support may stop, or maybe a better language may appear. But even if the development of Nim should stop someday, you will still be able to use it, and many concepts that you may have learned with Nim can be used with other modern languages too.

Is Nim a good choice as first language for a Beginner?

When you use C as your first language, you may learn well how computers really work, but the learning experience is not that nice, progress is slow and C lacks many concepts of modern programming languages. C++, Rust, or Haskell are really too difficult for beginners. So currently many start with Python. While you can learn high-level concepts well with Python, and you get useful results fast, you learn not much about the internal working of computers. So you may never understand why your code is slow and consumes so many resources, and you will have no idea how to improve the program or how you could run it successfully on restricted hardware. It’s like learning to drive a car, without any knowledge about how a combustion engine, the transmission, or the brakes really work. Nim has none of these restrictions, as we have high-level concepts available like in Python, but we have access to low-level stuff too, so we can really understand the internal workings if we want. Learning resources for Nim are still not that good as for mainstream languages, but there exist some nice tutorials already, and hopefully, this book will help beginners also a bit.

Is Nim really a good teaching language?

Generally yes, in the same way as Pascal was in the 1980s, and Modula/Oberon was at the end of the last century. But Nim still has the same problems as the wirthian languages: They do not really help with finding a job. When we teach the kids some JavaScript or C, they may find at least a simple employment when they have to leave the intended education path early for some reason. With niche languages this is unfortunately not the case, so teachers should know about their responsibility. And of course, teaching against the interests of the kids makes not much sense. When they want to learn some JavaScript to make some visual effects or whatever easily, then it is hard to teach another language which may not be immediately available on the PC at home or their smartphone.

So is Nim really the best start for me?

Maybe not. When you intend to learn a programming language today and make a great video game tomorrow, then definitely not. This is just not possible. While there are nice libs for making games with Nim already available, there exist easier solutions in other languages. With some luck, you may find some source code for that languages so that you can patch a few strings and perhaps modify some colors and the background music and call it your game.

After learning Nim, will I still have to learn other programming languages?

Nim is a quite universal language, so it is a good candidate for someone who intends to learn only one single language. But of course, it is always a good idea to learn a few other languages later. Generally, we can not really avoid learning C, as so much C code exists worldwide. Most algorithms that have ever been invented are available as a C implementation somewhere, and most libraries are written in C or have at least a C API, which you can use from other languages including Nim. As C is a small language without difficult language constructs, some minimal C knowledge is typically sufficient to convert a C program to another language. Often that conversion process is supported by tools, like the Nim c2nim tool. So learning some C later is really a good idea, and when you have some basic understanding of Nim and CS in general, learning some C is an easy task. Learning C before Nim would be an option still, as for C more learning resources exists. So, years ago, some people recommended learning C or Python before Nim. But Nim has enough learning resources now, so we recommend indeed starting with Nim directly.

Why should I not use Nim?

Perhaps it is just not the ideal solution for you. A racing bicycle or a mountain bike are both great devices, but for cycling a few hundred meters to the baker’s shop both may be not the perfect solution. A plain old bicycle would do better. Even as Nim seems to join the benefits of a racing bicycle and a mountain bike well — high performance and robust design — and is not expensive, it is just not the optimal solution for everybody. People who write only tiny scripts and have not to care about performance, can continue using Python. People who are only interested in special applications, maybe only in web development or only in tiny 8-bit microcontrollers, may not really need Nim. Nim can do this and much more well, but for special use cases better suited languages may still exist. And someone who has managed to learn C++ really well over a period of many years may decide to continue with C++ also. Currently, another possible reason for not using Nim can be missing libraries. When you require some important libraries for your project, and these are currently not available for Nim, this can be of course a big problem in the case that you do have not the skills or the time to write them from scratch or at least create high-level bindings to a C library.

Our first Nim Program

To keep our motivation, we will present the first tiny Nim program now. Actually, we should have delayed this section until we have installed the Nim compiler on our computer, but we can already run and test the program by just copying it into one of the available Nim online playgrounds like

In the section What is an Algorithm? we described an algorithm to sum up the first 100 natural numbers. Converting that algorithm into a Nim program is straightforward and results in the text file below. You can copy it into the playground and run it now if you want. The program is built using some elementary Nim instructions, for which we will give only a very short description here. Everything is explained in much more detail in the next part of this book.

var sum: int
var i: int
sum = 0
i = 0
while i < 100:
  inc(i, 1)
  inc(sum, i)
echo sum

We write Nim programs with an editor tool in the form of plain text files, and you will learn how to create them soon. We call these text files the source code of the program. The source code is the input for the compiler. The compiler processes the source code, checks it for obvious errors, and then generates an executable file, which contains the final CPU instructions and can be run. Executable files are sometimes called executables or binary files. The term binary is misleading, as all files on computers are indeed stored as binary data, but the expression "binary" is used to differentiate the executable program from text files like the Nim source code which we can read, print, and edit in an editor. Don’t try to load the executable files generated by the Nim compiler into a text editor, as the content is not plain text, but numeric machine code that may confuse the editor. On the Windows OS, executable files typically get a special name extension .exe, but on Linux, no special name extensions are used.

Nim source code files are processed by the Nim compiler from the top to the bottom, and for the generated executable the program execution also starts in principle at the top. But for the program execution there exist some exceptions, e.g. program code enclosed in functions is not immediately executed where it appears in the program source code file, but later when the function is called. And the program execution is not a linear process — we can use conditional expressions to skip parts of the program, or various loop constructs to repeat the execution of some program segments. Actually, the program execution in Nim is more similar to languages like Python or Ruby than to the C language: A C program always needs a main() function with exactly this name, and the execution of a C program always starts with a compiler-generated call of this function.

Elementary entities of computer programs are variables, which are basically named storage areas in the computer. As Nim is a compiled and statically-typed language, we have to declare each variable before we can use it. We do that by choosing a meaningful name for the variable and specifying its data type. To tell the compiler about our intention to declare a variable, we start the line with the var keyword, followed by the chosen name, a colon, and the data type of our variable. We have to put at least one space character between the var keyword and the name of the variable, to allow the compiler to recognize the two separate entities. Usually, we also put a space after the colon, that separates the variable name from its data type. But this is only a convention to improve the readability of the source code, for the compiler the colon already separates the variable name from the data type. The first line of our program declares a new variable named sum of data type int. Int is short for integer and indicates, that our variable should be able to store negative or positive integer numbers. The var at the start of the line is a keyword. Keywords are reserved symbols that have a special meaning for the compiler. Var indicates that we want to introduce a new variable. The compiler will recognize that and will reserve a memory location in the RAM of the computer which can store the actual value of the variable.

The second line is nearly identical to the first line: we declare another variable, again with int type and plain name i. Variable names like i, j, k are typically used when we have no idea for a meaningful name and when we intend to use that variable as a counter in a loop.

In the lines 3 and 4 of our program, we initialize the variables, that is, we give them a well-defined initial value. To do that, we use the = operator to assign a value to the variable. Operators are special symbols like +, -, *, or / to indicate our desire to do an addition, a subtraction, a multiplication, or a division. Note that the = operator is used in Nim like in many other programming languages for assignment, and not like in traditional mathematics as an equality test. The reason for that is, that in computer programming, assignments occur more frequently than equality tests. Some early languages like Pascal used the compound := operator for assignment, which may be closer to mathematics use, but is more difficult to type on a keyboard and looks not too nice for most people. An expression like x = y assigns the content of variable y to x, that is, x gets the value of y, the former value of x is overwritten and lost, and the content of y remains unchanged. After that assignment, x and y contain the same value. In the above example, we do not assign the content of a variable to the destination, but instead, use a literal numeric constant with the value 0. When the computer has executed lines 3 and 4 the variables sum and i each contain the start value 0. When we use the = operator for an assignment, we usually put a space character on both sides of the operator, but this is only a convention to improve the readability of the source code, and not really necessary. Actually, convention is to put a space on both sides of most Nim infix operators whenever we use them, this includes the arithmetic operators, the assignment operator, or relational operators like < or >. And when we use a colon or a semicolon to separate two entities from each other, we usually put also a space after the punctuation character, in the same way as we would do it in ordinarily text files.

Line 5 of our code example is much more interesting: it contains a while condition. The line starts with the term while, which is again a reserved keyword, followed by the logical expression i < 100 and a colon. An expression in Nim is something that has a result, as a math expression as 2 + 2 which has the numeric result 4 of type integer. A logical expression has no numerical result, but a logical (boolean) one, which can be true or false. The logical expression i < 100 depends on the actual content of variable i. The two lines following the line with the while keyword are each indented by two spaces, meaning that these lines start with two spaces more than the line before. This form of indentation is used in Nim (and Python) to indicate blocks. Blocks are grouped statements. The complete while loop consists of the line containing the while keyword followed by a block of statements. The block after the while condition is executed as long as the while condition evaluates to the logical value true. For the first loop iteration i has the initial value 0, the condition i < 100 evaluates to the boolean value true, and the block after the while condition is executed for the first time. In this block, we have the inc() instruction. Inc is short for increment. Inc(a, b) increases the value of variable a by b, b remains unchanged. So in the above block, i is increased by one, and after that, sum is increased by the current value of i. So when that block has been executed for the first time, i has the value 1 and sum also has the value 1. At the end of that block, execution starts again at the line with the while condition, now testing the expression i < 100 with i containing the value 1. Again it evaluates to true, the block is executed again, i gets the new value 2, and sum gets the value 3. This process continues until i has the value 100, so the condition i < 100 evaluates to false and execution proceeds with the first instruction after the while block. That instruction is an echo statement, which is used in Nim to write values to the terminal or screen of the computer. Some other languages use the term print or put instead of echo. You may still wonder about the colon that terminates line five with the while condition. That colon is (only) a marker to indicate the end of a conditional statement.

Don’t worry if you have not understood much of this short explanation, we will explain all that in much more detail later.

If you should decide to try the above program, maybe on a playground internet page or already on your local computer, then it is best to copy the source code verbatim instead to type it in from scratch, as for beginners tiny typos can generate a lot of trouble. For the case that you should decide to type it in with your keyboard, you should try to type it exactly as displayed above. All the program code should start directly at the first column, but the two lines after the while keyword should start with two spaces. This strict indentation is used in Nim and some other programming languages like Python or Haskell to structure the program code and to mark the extent of code blocks. Some other programming languages like C do a similar alignment of the source code for readability, but that alignment is ignored by the C compiler — instead blocks have to be enclosed in curly braces {}. Note that you have to do the indentation really with spaces, as Nim does not accept tabulator characters in its source files. Also note that the Nim compiler does distinguish between words starting with a lower or an upper case letter. Nim keywords are written always in lower case, and when we define a variable as sum then we should always refer to it in exactly this notion.[11] Also note that spaces in the Nim source code are important and can change the semantic: While in C spaces are mostly only used to separate distinct symbols, in Nim spaces have some more functionality. For instance, in mathematically expressions, a - b or a-b is both a valid subtraction in the case when a and b both have a numeric type for which an infix subtraction operator is defined, but the code segment a -b may give us an error message from the compiler. The reason is, that in this case, the - sign is directly attached to b but separated from a by at least one space. In this case, the Nim compiler would interpret the - sign as a unary operator attached to b. Even in the case that such a unary - may have been defined before, then the operands a and b would be not separated by an infix operator, which is an invalid syntax in Nim. An expression like a - -b would be a valid syntax instead — unary minus attached to b, and a and (-b) separated by an infix - operator. In this example, we have learned already that the same symbol can have a different meaning in the Nim language, depending on the context. For operators or functions, this is called overloading, which most modern programming languages use. This sensitivity to the asymmetric use of spaces applies also to the less than operator that we used in the above example: a < b or a<b is the infix notation that we generally intend for a comparison operation, while a <b would be mostly invalid code. For infix operators, we typically put a space on each side, as this improves readability, but it is not really needed and so some people do not insert these spaces. Unary operators, like the unary - sign, should always precede a variable or a literal without a space.

All this may sound a bit complicated, and for beginners, the compiler error messages about these formatting rules may not be always fully clear. But finally, it is just how we would write the code with paper and pencil, and after the initial learning phase, you just will do it right without thinking about it.

Note that you can easily verify the result of our tiny program: Instead of summing up the first 100 natural numbers, we can just sum up 50 pairs, built from the first and last summand, from the second and the one before the last, and so on. The sum of each pair is always 101, so for the sum of fifty pairs, we get 50 * 101 = 5050. This trick is attributed to the famous German mathematician Johann Carl Friedrich Gauss (1777 – 1855), who should have used it as a young schoolboy to solve this task given by a teacher.[12]

Binary Numbers

When we write numbers in ordinary life we typically use the decimal system with base 10 and the 10 available digits 0, 1, …​ 9. To get the value of a decimal number, we multiply each digit with powers of 10 depending on the position of the digit and sum the individual terms. The rightmost digit is multiplied with 10^0, the next digit with 10^1, and so on. A literal decimal number like 7382 has then the numerical value 2 * 10^0 + 8 * 10^1 + 3 * 10^2 + 7 * 10^3. We have used here the exponential operator ^ — with 10^3 = 10 * 10 * 10. Current computers use binary representation internally for numbers. Generally, we do not care much about that fact, but it is good to know some facts about binary numbers. Binary numbers work nearly identically to decimal numbers. The distinction is that we have only two available digits, which we write as 0 and 1. A number in binary representation is a sequence of these two digits. Like in the decimal system, the numerical value results from the individual digits and their position: The binary number 1011 has the numerical value 1 * 2^0 + 1 * 2^1 + 0 * 2^2 + 1 * 2^3, which is 11 in decimal notation. For binary numbers, the base is 2, so we multiply the binary digits by powers of two. Formally, the addition of two binary numbers works as we know it from the decimal systems: we add the matching digits and take carry into account: 1001 + 1101 = 10110 because we start by adding the two least significant digits of each number, which are both 1. That addition 1+1 results in a carry and result 0. The next two digits are both zero, but we have to take the carry from the former operation into account, so the result is 1. For the next position, we have to add 0 and 1, which is just 1 without a carry. And finally, we have 1 + 1, which results in 0 with a carry. The carry generates one more digit, and we are done. In the decimal system with base 10, a multiplication with 10 is easily calculated by just shifting all digits one place to the left and writing a 0 at the now empty rightmost position. For binary numbers it is very similar: a multiplication by the base, which is two in the binary system, is just a shift left, with the rightmost position getting digit 0.[13]

In the binary system, we call the digits typically bits, and we number the bits from right to left, starting with 0 for the rightmost bit — we say that the binary number 10010101 is an 8-bit number because writing that number in binary representation needs 8 digits. Often we imagine the individual bits as small bulbs, a 1 bit is imagined as a lit bulb, and a 0 bit is imagined as a dark bulb. For lit bulbs we say also that the bit is set, meaning that in the binary number 10010101, bits 0, 2, 4, and 7 are set, and the other bits are unset or cleared.

Groups of 8 bits are called a byte, and sometimes 4 bits are called a nibble.

One, two, four, or 8 bytes are sometimes called a word, where a word is an entity that the computer can process in one single instruction. When we have a CPU with 8-byte word size, this means that the computer can for example, add two variables, each 8 byte in size, in one single instruction.

Let us investigate some basic properties of binary numbers. Let us assume that we have an 8-bit word (a byte). An 8-bit word can have 2^8 different states, as each bit can be set or unset independently of the other bits. That corresponds to numbers 0 up to 255 — we assume that we work with positive numbers only for now, we will come to negative numbers soon. An important property of binary numbers in computers is the wrapping around, which is a consequence of the fact that we have only a limited set of bits available to store the number. So when we continuously add 1 to a number, at some point all bits are set, which corresponds to the largest number that can be stored with that number of bits. When we then add again 1, we get an overflow. The run-time system may catch that overflow, so we get an overflow error, or the number is just reset to zero, as it may happen in our car when we manage to drive one million miles, or when the ordinary clock jumps from 23:59 to 00:00 of the next day. A useful property of binary numbers is the fact that we can easily invert all bits, that is, replace set bits with unset ones and vice versa. Let us use the prefix ! to indicate the operation of bit inversion, then !01001100 is 10110011. It is an obvious and useful fact that for each number x we get a number with all bits set when we add x and !x. That is x + !x = 11111111 when we consider an 8 bit word. And when we ignore overflow, then it follows that x + !x + 1 = 0 for each number x. That is a useful property, which we can use when we consider negative numbers.

Now, let us investigate how we can encode negative numbers in binary form. In the binary representation, we have only two states available, 0 or 1, a set bit or an unset bit. But we have no unitary minus sign. We could encode the sign of a number in the topmost bit of a word — when the topmost bit is set, that indicates that the number is regarded as negative. Generally, a modified version of this encoding is used, called two’s complement: a negative number is constructed by first inverting all the bits — a 0 bit is transferred into a 1 bit and vice versa — and finally the number 1 is added. That encoding simplifies the CPU construction, as subtraction can be replaced by addition in this way:

Consider the case that we want to do a subtraction of two binary encoded numbers. The operation has the symbolic notation A - B for arbitrary numbers A and B. The subtraction is by definition the inverse operation of the addition, that is A + B - B = A for each number A and B, or in other words, B - B = 0 for each number B.

Assume we have a CPU that can do additions and that can invert all the bits of a number. Can we do subtraction with that CPU? Indeed, we can. Remember the fact that for each number X X + !X + 1 = 0 as long as we ignore overflow. If that relation is true for each number, then it is obviously true for each B in the expression A - B, and we can write A - B = A + (B + !B + 1) - B = A + (!B + 1) when we use the fact that in mathematics addition and subtraction is associative, that is we can group the terms as we want. But the term in the parenthesis is just the two’s complement, which we get when we invert all bits of B and add 1. So to do a subtraction we have to invert the bits of B, and then add A and !B and 1 ignoring overflow. That may sound complicated, but a bit inversion is a very cheap operation in a CPU, which is always available, and adding 1 is also a very simple operation. The advantage is that we do not need separate hardware for the subtraction operation. Typically, subtraction in this way is not slower than addition because the bit inversion and the addition of 1 can be performed at the same time in the CPU as an ordinary addition.

From the equation above, indicating A - B = A + (!B + 1) it is obvious that we consider the two’s complement (!B + 1) as the negative of B. Note that the two’s complement of zero is again zero, and two’s complement of 00000001 is 11111111. All negative numbers in this system have a bit set to 1 at the leftmost position. This restricts all positive numbers to all the bit combinations where the leftmost bit is unset. For an 8-bit word, this means that positive numbers are restricted to the bits 00000000 to 01111111, which is the range 0 to 127 in decimal notation. The two’s complement of decimal 127 is 10000001. Seems to be fine so far, but note that there exists also the bit pattern 10000000, which is -128 in decimal. For that bit pattern, there exists no positive value. If we try to build the two’s complement of that bit pattern, we would get the same pattern again. This is an asymmetry of two’s complement representation, which can not be avoided. It generally is no problem, with one exception. We can never invert the sign of the smallest available integer; that operation would result in a run-time error.[14]

Summary: when we work only with positive numbers, we can store in an 8-bit word, which is called a byte, numbers from 0 up to 255. In a 16-bit word, we could store values from 0 up to 2^16 - 1, which is 65535. When we need numbers that can be also negative, we have for 8-bit words the range from -128 to 127 available, which is -2^7 up to 2^7 - 1. For a signed 16-bit word, the range would be -2^15 up to 2^15 - 1.

While we can work with 8 or 16-bit words, for PC programming the CPU usually supports 32 or 64-bit words, so we have a much larger number range available. But when we program microcontrollers or embedded devices we may indeed have only 8 or 16-bits words available, or we may use such small words size intentionally on a PC to fit all of our data into a smaller memory area.

One important note at the end of this section: whenever we have a word with a specific bit pattern stored in the memory of our computer, then we can not decide from the bit pattern directly what type of data it is. It can be a positive or a negative number, but maybe it is not a number at all but a letter or maybe something totally different. As an example, consider this 8-bit word: 10000001. It could be 129 if we have stored intentionally positive numbers in that storage location, or could be -127 if we intentionally stored a negative value. Or it could be not a number at all. Is that a problem? No, it is not as long as we use a programming language like Nim which uses static typing. Whenever we are using variables, we declare their type first, and so the compiler can do bookkeeping about the type of each variable stored in the computer memory. The benefit is, that we can use all the available bits to encode our actual data, and we do not have to reserve a few bits to encode the actual data type of variables. For languages without static typing, that is not the case. In languages like Python or Ruby, we can use variables without a static type, so we can assign whatever we want to them. That seems to be comfortable at first, but can be confusing when we write larger programs and the Python or Ruby interpreter has to do all the bookkeeping at run-time, which is slow and wastes memory for the bookkeeping.

To say it again in other words: for deciding if an operation is valid, it is generally sufficient to know the data type of the operands only. We do not have to know the actual content. The only exception is if we invert the sign of the most negative integer number or if we do an operation with causes an overflow, as there are not enough bits available to store the result — we may get a run-time error for that case.[15] In a statically-typed language, each variable has a well-defined type, and the compiler can ensure at compile time that all operations on that variable are valid. If an operation is not valid, then the compiler will give an error message. Then when these operations are executed at run-time they are always valid operations, and the actual content, like the actual numeric value, does not matter.

Hexadecimal Numbers

This number type with base 16 is by far not as important as the binary numbers, and it has not really a technical justification to exist, but you may get in touch with these numbers from time to time. Hexadecimal numbers are mostly a legacy from the early days of computers, where computer programming was done not in real programming languages but with numeric codes. To represent the 16 hexadecimal digits, the 10 decimal digits are supported by the characters 'A' .. 'F'. The most significant property of a hexadecimal digit is that it can represent four bits, a unit half of a byte, which is called sometimes a nibble. In old times, when it was necessary to type in binary numbers, it was sometimes easier to encode a nibble with a hexadecimal digit:

Decimal Binary Hexadecimal

0

0000

00

1

0001

01

2

0010

02

3

0011

03

4

0100

04

5

0101

05

6

0110

06

7

0111

07

8

1000

08

9

1001

09

10

1010

0A

11

1011

0B

12

1100

0C

13

1101

0D

14

1110

0E

15

1111

0F

The only location where we hear about hexadecimal characters again in this book should be when we introduce the character and string data types — there control characters like a newline character are sometimes specified in hexadecimal form like "\x0A" for a newline character.

Installation of the Compiler

We will not describe in too much detail how you can install the Nim compiler, because that strongly depends on your operating system, and because the installation instructions may change in the future. We assume that you have a computer with an installed operating system and internet access, and you are able to do at least very basic operations with your computer, such as switching it on, logging in, and opening a web browser or a terminal window. If that is not the case, then you really should ask someone for help with this basic step, and maybe for some more help for other basic tasks.

Detailed installation instructions are available on the Nim internet homepage at To visit and read that page Try to follow those instructions, and when they are not sufficient, then please ask at the Nim forum for help: https://forum.nim-lang.org/

If you are using a Linux operating system, then your system usually provides a package manager, which should make the installation very easy.

For example, for a Gentoo Linux system, you would open a root terminal and simply type emerge -av nim. That command would install Nim, including all necessary dependencies, for you. It may take a few minutes as Gentoo compiles all packages fresh from source code, but then you are done. Similar commands exist for most other Linux distributions. This installation by a package manager installs Nim system-wide, so all users of the computer can now use Nim.

Another solution, which is preferable when you want to ensure that you get the most recent Nim compiler, is compiling directly from the latest git sources. That process is also easy and is described here: https://github.com/nim-lang/Nim. But before you can follow those instructions, you have to ensure that the git software and a working C compiler are available on your computer.

Creation of Source Code Files

Nim source code, as most source code of other programming languages, is based on text files. Text files are documents saved on your computer that contain only ordinary letters, which you can type on your keyboard. No images or videos, no HTML content with fancy CSS styling. Generally, source code should contain only ordinary ASCII text, that is, no umlauts or Unicode characters.

To create source code, we typically use a text editor, which is a tool designed for creating and modifying plain text files. If you do not have a text editor yet, you may also use a word processor for writing some source code, but then you have to ensure that the file is finally saved as plain ASCII text. Editors typically support syntax highlighting, that is keywords, numbers, and such are displayed with a unique color or style to make it easier to recognize the content. Some editors support advanced features like checking for errors while you type the program source code.

A list of recommended editors is available at https://nim-lang.org/faq.html

If you do not want to use a special editor now, then for Linux Gedit or at least Nano should be available. For Windows, maybe something like Notepad.

Typically, we store our Nim source code files in their own directory, which is a separate section of your hard disk. If you work on Linux in a terminal window, then you can type

cd
mkdir mynimfiles
cd mynimfiles
gedit test.nim

You type these commands in the terminal window and press the return key after each of the above lines — that is, you type cd on your keyboard and then press the return key to execute that command. The same for the next three commands. What you have done is this: you went to your default working area (home directory), then created a subarea named mynimfiles, then you went into that subarea, and finally you launched the gedit editor — the argument test.nim tells gedit that you want to create a new file called test.nim. If gedit is not available, or if you work on a computer without a graphical user interface, then you may replace the gedit command with nano. While gedit opens a new window with a graphical interface, nano opens only a very simple interface in the current terminal. An interesting editor without a GUI is Vim or NeoVim. That is a very powerful editor, but it is difficult to learn, and it is a bit strange as you have a command mode and an ordinary text input mode available. For NeoVim there is very good Nim support available.

If you do not want to work from a terminal, or if you are using Windows or macOS, then you should have a graphical user interface that enables you also to create a directory and launch an editor.

When the editor is opened, you can type in the Nim source code from our previous example and save it to a file named test.nim. Then you can terminate the editor.

Note that the return key behaves differently in editors than in the terminal window: In the terminal window, you type in a command and finally press the return key to "launch" or execute the command. In an editor, the return key is not that special: if you press ordinary keys in your editor, then that key is inserted and the cursor moves one position to the right. And when you press the return key, then an invisible newline character is inserted and the cursor moves to the start of the next line.

Launching the compiler and running the program

If you are working from a Linux terminal, then you can type

ls -lt
cat test.nim

That is, you first show the content of your directory with the ls command, and then display the content of the Nim source code file that you just have typed in with the cat command.

Now type

nim c test.nim

That invokes the Nim compiler and instructs it to compile your source code. The "c" letter is called an option or a sub-command, it tells the Nim compiler to compile your program and to use the C backend to generate an executable.

The compiler should display nearly immediately a success message. If it displays some error messages instead, then you launch Gedit or Nano again, fix your typing error, save the modified file and call the compiler again.

Finally, when the source text is successfully compiled, you can run your program by typing

./test

In your terminal window, you see a number now, which is the sum of the numbers 1 to 100.

You may wonder why you have to type the prefix ./ in front of the name of your generated executable program, as you can launch most other executables on your computer without such a prefix. The prefix is generally needed to protect you and your computer from erroneously launching a program in the current directory while you intended to launch a system command. Imagine you downloaded a zip file from the internet, extract it, cd into the extracted directory, and type ls to see the directory content. Imagine now that the directory contains an executable named ls, which is executed instead of system ls. That foreign ls command may damage your system. So to execute non-system commands, you generally have to use the prefix ./ where the period refers to the current directory. Of course, you can install your own programs in a way that you don’t need such a prefix anymore — just ask your Mom or Grandma if you don’t know yourself already.

If you have not managed to open a terminal where you can invoke the compiler — well, maybe then you should install some advanced editors like VS-Code. They should be able to launch the compiler and run the program from within the editor directly.

The command

nim c test.nim

is the most basic compiler invocation. The extension .nim is optional, the compiler can infer that file extension. This command compiles our program in default debug mode, it uses the C compiler back end and generates a native executable. Debug mode means that the generated executable contains a lot of checks, like array index checks, range checks, nil dereference checks, and many more. The generated executable will run not very fast, and it will be large, but when your program has bugs, then the program will give you a meaningful error message in most cases. Only after you have tested your program carefully, you may consider compiling it without debug mode. You may do that with

nim c -d:release test.nim

nim c -d:danger test.nim

The compiler option -d:release removes most checks and debugging code and enables the backend optimization by passing the option "-O3" to the C compiler backend, giving a very fast and small executable file. The option -d:danger removes all checks, it includes -d:release. You should be aware that compiling with -d:danger means that your program may crash without any useful information, or even worse, may run, but contain uncatched errors like overflows and so may give you wrong results. Generally, you should compile your program with plain nim c first. When you have tested it well, and you may need the additional performance, you may switch to -d:release option. For games, benchmarks, or other uncritical stuff, you may try -d:danger.

There exist many more compiler options, you can find them explained in the Nim manual, or you may use the command nim --help and nim --fullhelp to get them displayed. One important new option is --gc:arc to enable the new deterministic memory management. You may combine --gc:arc with -d:useMalloc to disable Nim’s own memory allocator, this reduces the executable size and enables the use of Valgrind to detect memory leaks. Similar to --gc:arc is the option --gc:orc, which can deal with cyclic data structures. Finally, a very powerful option is --passC:-flto. This option is for the C compiler backend and enables link time optimization (LTO). LTO enables inlining for all procedure calls and can significantly reduce the final program size. For a recent Nim compiler version, instead of --passC:-flto also -d:lto can be used. We should mention that you can also try the C++ compiler backend with the cpp sub-command instead of the plain c command, and that you may compile with the CLang backend instead of the default GCC backend with the --cc:clang option. You can additionally specify the option -r to immediately run the program after a successful build. For testing small scripts, the compiler invocation in the form nim r myfile.nim can be used to compile and run a program without the generation of a permanent executable file. Here is an example of how we use all these options:

nim c -d:release --gc:arc -d:useMalloc --passC:-flto --passC:-march=native board.nim

In this example, we additionally pass -march=native to the C compiler backend to enable the use of the most efficient CPU instructions of our computer, which may result in an executable that will not run on older hardware. Of course, we can save all these parameters in configuration files, so that we don’t have to actually type them for each compiler invocation. You may find more explanations for all the compiler options in the Nim manual, or in later sections of this book, this includes the options for the JavaScript backend.

Part II: The Basics

In this part of the book, we will introduce the most essential constructs of the Nim programming language, like statements and expression, conditional and repeated code execution, functions and procedures, iterators, templates, exceptions, and we will discuss various basic data types including the container types array, sequence, and string.

Declarations

We can declare constants, variables, procedures, or our custom data types. Declarations are used to give information to the compiler, for example, about the name and data type of a variable that we intend to use.

We will explain the type and procedure declarations in later sections. Currently, only constant and variable declarations are important.

A constant declaration in its simplest form maps a symbolic name to a value, like

const Pi = 3.1415

We use the reserved word const to tell the compiler that we want to declare a constant which we have named Pi, and we assign it the numeric decimal value 3.1415. Nim has a small set of reserved words like var, const, proc, while, and others, to tell the compiler that we want to declare a variable, a constant, a procedure, or that we want to use a while loop for some repeated code execution. Reserved words in Nim are special symbols that have a special meaning for the compiler, and we should avoid using these symbols as names for other entities like variables, constants, or functions, as that would confuse the compiler. The = is the assignment operator in Nim, it assigns the value or expression on the right side of it to the symbol on the left. You have to understand that it is different from the equal sign we may use in mathematics to express an equality relation. Some languages like Pascal initially used the compound operator := for assignments, but that is not easy to type on the keyboard and looks a bit angry for sensible people. And source code usually contains a lot of assignments, so the use of = makes some sense. For the actual equality test of two entities, which is not used that often, we use the compound == operator in Nim, as in most other programming languages including C and Python. We call = an operator. Operators are symbols that perform some basic operation, like + for the addition of two numbers, or = for the assignment of a value to a symbol. Most operators are used as infix operators in between two arguments, as in the expression 2 * Pi to denote the multiplication of the constant Pi with the literal number 2, resulting in the floating-point value 6.29. But operators can also be used as unitary operators, like in -Pi to invert the sign of a numeric value. When we declare named constants, we must always assign a value immediately. That value can never change, but of course, we can use the named constant in expressions to create different values as in

const Pi = 3.14
const TwoPi = 2 * Pi
const MinusPi = -Pi

We mentioned in part I of the book already, that we usually put a space on both sides of an operator when we use it in an infix notation between two operands, but this is only a convention to improve the readability of the source code. We also mentioned that in Nim spaces can in some cases change the interpretation of expression: Nim follows the traditional notations of handwritten text, e.g a + -b is very different from a+-b. We will discuss these notations in later sections of the book in more detail.

With the above constant declaration, we can use the symbol Pi in our program’s source code and don’t have to remember or retype the exact sequence of digits. Using named constants like our Pi above makes it easy to modify the value — if we notice that we need more precision, we can look up the exact value of Pi and change the constant declaration at one place in our source code, we don’t have to search for the digit sequence 3.14 in all our source code files.

For numeric constants like our Pi value, the compiler will do a substitution in the source code when the program is compiled, so where we write the symbol Pi the actual numeric value is used.

For constant declarations, it must be possible to determine its value at compile time. Expressions assigned to constants can contain simple operations like basic math, but some function calls may not be allowed.

Variable declarations are more complicated, as we ask the compiler to reserve a named storage location for us:

var velocity: int

Here we put the reserved keyword var at the beginning of the line to tell the compiler that we want to declare a variable, then we give our chosen name for that variable followed by a colon and the data type of the variable. The int type is a predefined numeric data type indicating a signed integer. The storage capacity of an integer variable depends on the operating system of your computer. On 32-bit systems, 32 bits are used, and on 64-bit systems, 64 bits are used to store one single integer variable. That is enough for even large signed integer numbers: the range is -2^31 up to 2^31 - 1 for 32-bit systems and -2^63 up to 2^63 - 1 for 64-bit systems.

For variables, we generally use lower case names, but names of constants may start with an upper case letter.

Variables declared by use of the var keyword are some form of a simple container, they store a value, which we can access or change later. We can assign to the variable immediately an initial value when we declare it, in a similar way as we have to do it for constants, or we assign the value later. As long as we do not assign an actual value, the variable has the default value, which is zero for numeric variables:

var start: int
var stop: int
var delta: int = 3
stop = 10 * start + 1

In lines one and two, we declare two variables called start and stop, which initially have the default integer value zero. In line three, we declare a different integer variable called delta, to which we assign an initial value 3. And finally, in line four, we assign to variable stop an integer expression. For variable declarations Nim offers some more variants, which we will discuss soon: We can use type inference, when we immediately assign an initial value, we can use var sections to introduce multiple variables without the need to type the var keyword multiple times, we can list multiple names of the same data type in front of the colon separated by commas, or we may use the let keyword to declare immutable variables.

In some Nim documentation and in this book, the terms declaration and definition may be used alternately, which is not fully correct. Precisely, a declaration is a statement announcing that something exists, while a definition is a more detailed description. In the C programming language, we differentiate between a function declaration, which describes only the name of the function and the number and data types of its parameters, and a function definition, which also has to specify the names of the function parameters as well as the source code of the function body. In Nim, function declarations are not used that often, as they are only really needed when two functions call each other. For that case, we declare the first function, so that we can already use it in the definition of the other function before we finally also define the first function. For other entities like constants, variables, data types, or modules, a distinction between the terms declaration and definition makes not that much sense, that is why we may use both terms alternately.

Statements

Statements, or instructions, are a core component of Nim programs: they tell the computer what it shall do. Often statements are procedure calls, like the call of the echo() or inc() procedure, which we have already seen in part I of the book. We will learn what procedures exactly are in later sections. For now, we just regard procedures as entities that perform a well-defined task for us when we call (or invoke) them. We call them by just writing their name in our source code file, followed by a list of parameters, also called arguments. When we write echo 7 then echo is the procedure which we call, and 7 is the argument, an integer literal in this case. When the parameter list has more than one argument, then we separate the arguments with a comma each, and generally, we put an optional space after that comma. The effect of our procedure call is that the decimal number 7 is written to the terminal when we run the program after compilation. While in languages like C the parameter list has to be always enclosed in brackets, in Nim we can frequently leave the brackets out, which is called command invocation syntax.

const SquareOfFive = 5 * 5
echo(5 * 5, SquareOfFive) # ordinary procedure call
echo 5 * 5, SquareOfFive # command invocation syntax

The command invocation syntax is often used with the echo() procedure, or when a procedure has only one single argument. For multiple arguments, or when the argument is a complicated expression, the use of brackets may be preferable. Some coding styles of other programming languages, like C, sometimes put a space between the procedure name and the opening bracket. For Nim, we should not do that, the reason will become clear when we later explain the tuple data type. A few procedures have no parameters at all. When we call these functions, we have to use always the syntax myProc() with an empty pair of brackets, to make it for the compiler clear that we want to call that function. res = myProc() assigns the result of the proc call to res, while res = myProc would assign the proc itself to res, which is very different.

A special form of procedures are functions, which are procedures that perform operations to return a value or a result. In mathematics, sin() or cos() would be functions — we pass an angle as an argument and get the sine, or cosine as a result.

Let’s look at this minimal Nim program:

var a: int
a = 2 + 3
echo a
echo(cos(0) + 2)

The Nim program above consists of a variable declaration and three statements: in the first line, we declare the variable we want to use. In the next line, we assign the value 2 + 3 to it, and finally, in line 3 we use the procedure echo() to display the content of our variable in the terminal window. In the last line, we use again the echo procedure with an ordinary parameter list enclosed in brackets. The parameter list has only one parameter, which is the sum of a function call and the literal value 2. Here the compiler would first call cos(0) and then add the literal value 2 to that result before finally the sum is passed to the echo proc to print the value.[16]

Nim programs are generally processed from top to bottom by the compiler, and when we execute the program after successful compilation, then it also executes from top to button. A consequence of this is that we have to write the lines of the above program exactly in that order. If we moved the variable declaration down, then the compiler would complain about an undeclared variable because the variable is used before it has been declared. If we exchanged lines 2 and 3, then the compiler would be still satisfied, and we would be able to compile and run the program. But we would get a very different result because we would first try to display the value of variable a, and later assign a value to it.

When we have to declare multiple constants or variables, then we can use a block, that is we write the keyword var or const on its own line, followed by the actual declarations like in

const
  Pi = 3.1415
  Year = 2020
var
  sum: int
  age: int

These blocks are also called sections, e.g. const section or var section, as known from the wirthian languages. Note the indentation — the lines after const and var start with some space characters, so they build a block that allows the compiler to detect where the declaration ends. Typically, we use two spaces for each level of indentation. Other numbers would work also, but the indentation scheme should be consistent. Two spaces are the general recommendation, as it is clearly recognizable for humans in the source code, and because it doesn’t waste too much space, that is, it would not generate long lines which may not fit onto the screen.

Also note, that in Nim we generally write each statement onto its own line. The line break indicates to the compiler that the statement has ended. There are a few exceptions — long mathematical expressions can continue on the next line (see the Nim manual for details). We can also put multiple statements on a single line when we separate them by a semicolon:

var a: int
echo a; inc(a) (1)
a = 2 * a + (2)
  a * a
1 two statements separated by a semicolon on a single line
2 a longer math expression split over multiple lines. An operator as the last character on a line indicates that the expression continues on the next, indented line.

We can also declare multiple variables of the same type in one single declaration, like

var
  sum, age: int

or we can assign an initial start value to a variable like in

var
  year: int = 1900

Currently, Nim allows also to initialize multiple variables with the same value:

var
  i, j: int = 1

Here, i and j would both get the initial value 1. Most people avoid this notation, as it is not that clear.

Finally, for variable declarations we can use type inference when we assign an initial start value, that is we can write

var
  year = 1900

The compiler recognizes in this case that we assign an integer literal to that variable, and so silently gives the variable the int type for us. Type inference can be comfortable but may make it harder for readers to understand the code, or the type inference may not always do exactly what we want. For example, in the above code, year gets the data type int, which is a signed 4 or 8-byte number. But maybe we would prefer an unsigned number or a number that occupies only two bytes in memory. So use type inference with some caution.

Note: For integral data, we mostly use the int data type in Nim, which is a signed type with a 4 or 8-byte size. It usually does not make sense to use many different integral types — signed, unsigned, and types of different byte sizes. Mixing them in numerical expressions can be confusing and potentially even decrease performance, because the computer may have to do type conversion before it can do the math operation. For unsigned types, another problem is that math operations on unsigned operands could have a negative result. Consider the following example, where we use a hypothetical data type "unsigned int" to indicate unsigned integers:

var a, b: unsigned int
a = 3
b = 7
a = a - b

The true result would be -4, but a is of unsigned type and can never contain negative content. So what should happen — an incorrect result or a program termination?

Related to variable declarations is the initial start value of variables. Nim clears for us all the bits of our variables when we declare them, that is, numbers always get the initial start value of zero if we do not assign a different value in the variable declaration.

In this declaration

var
  a: int = 0
  b: int

both variables get the initial value of zero.

There exists a variant for variable declarations which uses the let keyword instead of the var keyword. Let is used when we need a variable that only once gets a value assigned, while var is used when we want to change the content of the variable during program execution. Let seems to be similar to const, but in const declarations, we can use only values that are known at compile time. Let allows us to assign to variables values that are only available at program run time, maybe because the value is a result of a prior calculation. But let indicates, at the same time, that the assignment occurs only once, the content does not change later, during the program’s execution. We say that the variable is immutable. The use of the let keyword may help the human reader of the source code with understanding what is going on, and it may also help the compiler doing optimizations to get faster, or more compact code. For now, we can just ignore let declarations and use var instead — later, we may use let where appropriate, and the compiler will tell us when let will not work, and we have to use var.

The way how we declare constants, variables, types, and procedures in Nim is very similar as it was done in the wirthian languages Pascal, Modula, and Oberon. People coming from languages like C sometimes argue that C uses a shorter and better variable declaration of the form int velocity; instead of Nim’s var velocity: int. Indeed, that declaration is shorter in this case. And some people like it better that the data type is written first, they consider the data type more important than the name of the variable. That is a matter of taste, and the C notation would not work well for var/let/const distinction and for type declarations.

With what we have learned in this section, we can rewrite our initial Nim example from part I in this form:

const
  Max = 100
var
  sum, i: int
while i < Max:
  inc(i)
  inc(sum, i)
echo sum

In the code above, we declare both variables of type int in a single line and take advantage of the fact that the compiler will initialize them with 0 for us. And we use a named constant for the upper loop boundary. Another tiny fix is that we write inc(i) instead of inc(i, 1). We can do that because there exist multiple procedures with the name inc() — one which takes two arguments, and one which takes only one argument and always increases that argument by one. Procedures with the same name, but different parameter lists, are called overloaded procedures. Instead of inc(i) we could have written also i = i + 1 and instead of inc(sum, i) we could write sum = sum + i. That would generate identical code in the executable, so we can use whatever we like better.

Input and Output

We have already used the echo() procedure for displaying textual output in the terminal window. In the code examples of the previous sections, we always passed arguments of integer type to the echo() proc, and the echo() proc automatically converted the integer numbers to a textual sequence of decimal digits so that we could read it in the terminal. In the Nim programming language, text is a predefined, built-in data type that is called string. We will learn all the details of the string data type in the next section, for now, it is sufficient that it exists and that we can use the echo() proc to print text strings. The echo() proc has the ability to convert other data types like numbers or the boolean data type (true/false) automatically to human-readable text strings, so that we can read the output in the terminal. Recall that most data types are stored internally in our computer as bits and bytes, which have no true human-readable representation by default. Numbers, like most other data types stored in the computer, are actual abstract entities. We have learned already that all data in the computer is internally stored in binary form, that is, as a bit pattern of 0 and 1. But even that bit pattern is still an abstraction, we would require a procedure that prints a 0 for each unset bit and a 1 for each set bit to display the content of an internally stored number in binary form in the terminal or elsewhere. In the same way, we require a procedure to print an internally stored number as a human-readable sequence of decimal digits. Even text strings are internally stored as abstract bit patterns, and we need conversion procs to print the content for us as ordinary text. All that can be done by the echo() proc, but we do not care for the actual details at this point of the book.

For our further experiments, we may also want to be able to enter some user data in the terminal. As we do not know much about the various available data types and the procs that can be used to read them in, we will just present a procedure that can read in a text string that the user has typed in the terminal window. We use a function with the name readLine() for this task.

echo "Please enter some text"
var mytext = readLine(stdin)
echo "you entered: ", mytext

Note that you have to press the return key after you have entered your text.

The first line of our program shows how we can print a text literal string with the echo() proc. To mark text literals unambiguously and to separate them from other literals like numeric literals or from variables, the string literals have to be enclosed in quotation marks. In the second line of our example program, we use the readLine() function to read textual user input. Note that we call readLine() a function, not a procedure, to emphasize that it returns a value. The readLine() function needs one parameter to know from where it should read — from the terminal window or from a file, for example. The stdin parameter indicates that it should read from the current terminal window — stdin is a global variable of the system (io) module and indicates the standard input stream. Finally, in line 3 we use again the echo() procedure to print some text. In this case, we pass two arguments to echo(), a literal text enclosed in quotes, and then separated by a comma, the mytext variable. The mytext variable has the data type string. We used type inference in this example to declare that data type: the readLine() procedure always returns a string, the compiler knows that, so our mytext variable is automatically declared with type string. We will learn more about the data type string and other useful predefined data types in the next section.

Nim supports the method call syntax, which was earlier called Uniform Function Call Syntax in the D programming language. With that syntax, we can write procedure calls in the form a.f instead of f(a). We will discuss that syntax in more detail when we explain procedures and functions. For now, it is enough that you know about the existence of that syntax, as we may use it in some places in the following sections. For example, for the length of text strings, we generally write myTextString.len instead of len(myTextString). Both notations are fully equivalent.[17]

When you try the example code from above, you may want a variant of it that does read in the textual input not on its own line, but directly after the prompt like "What is your name: Nimrod". As the echo proc always writes a newline character after the last argument has been written, we have to use a different function to get the input prompt on the same line. We can use the write() proc from the system module for this. As write() can not only write to the terminal, but also to files, it needs an additional parameter that specifies the destination. We can pass the variable stdout from the system module to indicate that write() should write to our terminal window. Another desire of beginners is, generally, to have the ability to read single character input without the need to press additional the return key. For that, we can use the getch() function from the terminal module — that function waits (blocks) until a key is pressed and returns the ASCII character of the pressed key:

from std/terminal import getch

stdout.write("May you tell me your name: ")
var answer = readLine(stdin)
if answer != "no":
  echo "Nice to meet you, ", answer
echo "Press any key to continue"
let c = getch()
echo "OK, let us continue, you pressed key:", c

Don’t get confused by the fact that the first write() call and the following readline() call does not appear on the same line in our example. The actual format of our source code does in this case not influence the program output. We could write both function calls on a single line, separated by a semicolon. But that would make no difference for the program output. The significant difference beteen the code above is just, that write() prints the text, but does not move the cursor in the terminal window to the next line, while echo() moves the cursor to the next line when all arguments have been printed. We say that echo() prints automatically a '\n' character, which we call a newline character, after all the arguments have been printed.

Data types

Nim is a statically typed programming language, which indicates that all variables have a well-defined data type, and this data type does not change during program execution. Further, we say that Nim is a strongly typed language, which means that Nim does nearly no automatic type conversions when variables are assigned to each other or are used in expressions or as arguments in function calls. Automatic type conversion may appear nice at first, but it can easily introduce errors or decrease the performance of our programs.

The most fundamental data type — in real life and in computer science — are integer (whole) numbers. All other numeric data types, like fractional, floating-point, or complex numbers, and other fundamental types like the boolean type with its two values true and false, or character and text string types, can be represented as integers. For that reason, the early computers built in the 1950s as well as today’s tiniest microcontrollers work internally only with integer numbers. The integer data type is not only very important for arithmetic operations, but is also used as an index to access elements in containers like arrays, and the integer numbers are often interpreted as bit vectors to represent set-like data types. As all CPUs are able to do basic bit operations like setting or clearing individual bits, and as bit patterns map well to mathematical sets, set data types are well-supported by all CPUs, and so set operations are generally very efficient. Advanced computers built in the 1980s got support for the important class of floating-point numbers by special floating-point processors for fast numerical computations. These floating-point units are today typically integrated into the CPU, and GPUs can even process many floating-point operations in parallel, where the precision is typically restricted to ranges needed for games and graphics animation, that is 32 or even 16 bit. Modern CPUs have often also some form of support for vector data types to process multiple values in one instruction (SIMD, single instruction, multiple data).

None numeric types like characters or text strings are internally represented by integer numbers — in the C language the data type to present text strings is called char, but it is indeed only an 8-bit integer type that supports all the mathematical operations defined for ordinary integer types. In Nim and the wirthian languages, most math operations are not directly allowed for the char data type, which should prevent misuse and allows catching logical errors by the compiler.

Nim supports also some built-in homogeneous container types like arrays and sequences, and many built-in derived types like enumeration types, sub-ranges and slices, distinct types, and view types (experimental). The built-in inhomogeneous container types object and tuple, which allow grouping other types, are supported by a variant type container, which allows instances of that type to contain different child types at runtime. These inhomogeneous container types are similar to the struct and union types from the C programming language.

Other basic and advanced data types like complex and fractional numbers, types with arbitrary-precision arithmetic as well as hash sets and hash tables, dynamically linked list or tree structures are available through the Nim standard library or external packages. And of course, we are able to define our own custom data types with our own operators, functions, and procedures working on them.

Note that all the data types that are built into the language, like the primitive types int, float, or char, as well as the built-in container types like tuple, object, seq, and string, are written in lower case, while data types that are defined by the Nim standard library or that we define our self, by convention, starts with a capital letter like the CountTables type defined in the tables module. Some people may regard this as an inconsistency, others may say that in this way we can differentiate built-in types from types defined by libraries. At least we may agree that using capital notation for common types as in Int, Float, or String would be more difficult to type and would look not that nice.

Integer types

We have already used the int data type, which indicates a signed integer type of 4 or 8-byte size, depending on the operating system. The reason why the size of that type depends on the word size of the OS will become clear later, when we explain what references and pointers are.

Besides the int data type, Nim has some more data types for signed and unsigned integers: int8, int16, int32, and int64 are signed types with well-defined bit and byte size, and uint8, uint16, uint32, and uint64 are the unsigned equivalents. The number at the end of the type name is the bit size; we get the byte size when we divide that value by 8. Additionally, we have the type uint, which corresponds to int and has the same size, but stores unsigned numbers only. [18] Generally, we should try to use the int type for all integral numbers, but sometimes it can make sense to use the other types. For example, when you have to work with a large collection of numbers, you know that each number is not very big, and your RAM is not really that large, then you may decide, for example, to use int16 for all your numbers. Or when you know that your numbers will be huge and will not fit in a 4-byte integer, then you may use the int64 type to ensure that the numbers fit in that type even when your program is compiled and executed on a computer with a 32-bit OS.

For integer numbers, we have the predefined operators +, -, and * available for addition, subtraction, and multiplication. Basically, these operations work as we may expect, but we have to remember that we may get overflows. For signed integers, we get compile- or run-time errors in that case, while unsigned integers just wrap around, see the example at the end of this section. For the division of integers, we have the operators div, mod, and / available. The div operator does an integer division ignoring the remainder, mod is short for modulus and gives us the remainder of the division, and / finally is currently only predefined for the signed int type and gives us a fractional result of data type float. That type is introduced in the next section.

Remembering how div and mod behave when the divisor or dividend is negative can be confusing, and it may differ for other programming languages. You may find a detailed justified explanation for the concrete behavior in the Nim manual and on Wikipedia.

Integer division for positive and negative operants
Result of i div j
   -4 -3 -2 -1  0  1  2  3  4
-4  1  1  2  4    -4 -2 -1 -1
-3  0  1  1  3    -3 -1 -1  0
-2  0  0  1  2    -2 -1  0  0
-1  0  0  0  1    -1  0  0  0
 0  0  0  0  0     0  0  0  0
 1  0  0  0 -1     1  0  0  0
 2  0  0 -1 -2     2  1  0  0
 3  0 -1 -1 -3     3  1  1  0
 4 -1 -1 -2 -4     4  2  1  1

Result of i mod j
   -4 -3 -2 -1  0  1  2  3  4
-4  0 -1  0  0     0  0 -1  0
-3 -3  0 -1  0     0 -1  0 -3
-2 -2 -2  0  0     0  0 -2 -2
-1 -1 -1 -1  0     0 -1 -1 -1
 0  0  0  0  0     0  0  0  0
 1  1  1  1  0     0  1  1  1
 2  2  2  0  0     0  0  2  2
 3  3  0  1  0     0  1  0  3
 4  0  1  0  0     0  0  1  0

When performance matters, we generally should try to use the "CPU native" number type, which for Nim is the int type. And we should try to avoid using math expressions with different types, as the CPU may have to do type conversion in that case before the math operation can be applied. Adding two int8 types can on some CPUs be slower than adding two ints, because the CPU may have to size extend the operands before the math operation is performed. But this depends on the actual CPU, and there are important exceptions: Multiplying two ints would result in an int128 result if the int size is 64 bit, which can be slow when the CPU does not support that operation well. Another essential point to consider for maximum performance is the cache usage. If you are performing operations on a large set of data, then you may get a significant performance gain when large fractions of your data fit in the caches of your computer, as cache access is much faster than ordinary RAM access. So using smaller data types, i.e. int32 instead of Nim’s default int, which is int64 on a 64-bit OS, may increase performance in this special application.

When we use Nim on tiny microcontrollers, maybe even on 8-bit controllers like the popular AVR devices, it may be best to use only integers of well-defined size like int8.

When we write integer literal numbers, then we use generally our common decimal notation, as in var i = 100. To increase the readability for long number literals we can use the underscore character as in 1_000, that underscore character is just ignored by the compiler. We can also write integer literals in binary, octal, or hexadecimal notation. For that, we prefix the literal value with 0b, 0o, or 0x. The leading zero is necessary, and the next letter indicates a binary, octal, or hexadecimal encoding. But such integer literal notation is very rarely used.

More important is the actual size of integer literals, in particular when we use type inference. Ordinary integer literals have the int type, but integer literals not fitting in 32 bit have int64 type. We can also specify the type of integer literals by appending the literal with i8, i16, i32, or i64 for signed types and with u, u8, u16, u32, or u64 for unsigned types. We can separate the actual number and the suffix with a ' character, but that is not necessary for the integer literals.

var
  a = 100 # int literal in decimal notation
  b = 1234567890000 # int64
  c = 5'i8 # 8-bit integer
  d = 7u16 # unsigned integer with 2 byte size
  e = 0b1111 # ordinary integer in binary notation, value is 15 in decimal notation
  f = 0o77 # integer in octal notation, value is 7 * 8^0 + 7 * 8^1 in decimal notation
  g = 0xFF # integer in hexadecimal notation

echo g, typeof(g)

In arithmetic expressions, integer types of different sizes are generally compatible when all the types are signed or unsigned, e.g. in the example code from above we could write echo a + b + c, and typeof(a + b + c) is int64, that is the expression is propagated to the largest type of all the involved operands. But echo a + b + c + d would not compile, as for such a mix of signed and unsigned operands, it is not clear if signed or unsigned arithmetic should be used. Also note that even on a 64-bit OS, echo typeof(a) is typeof(b) would print false.

An important property of the current Nim implementation of A. Rumpf, used with the C backend, is the fact that unsigned integers do not generate overflow errors but simple wrap around:

var x: int8 = 0

while true:
  inc(x)
  echo x

The above code would print the numbers 0 up to 127 and then terminate program execution due to an overflow error. But when we change the data type to uint8, we would get a continuous sequence of the numbers 0 up to 255. After the value 255 is reached, the value wraps around to 0 again and the process continues. This behavior can lead to strange bugs and is one of the reasons why the Nim team generally recommends avoiding unsigned integers.

For compatibility with external libraries, Nim has also the integer types cint and cuint, which exactly match the C types int and uint when we compile for the C or C++ backend. For the JavaScript backend, the LLVM backend, or other backends, these types may be also available, for details you should consult the compiler documentation. For most operating systems and C compilers the int and uint types in C are 4 bytes in size, but there can be exceptions, so we better should not write code that depends on the actual byte size of the types. The Nim types cint and cuint are mainly only used for parameter lists of © library functions. To match other C integer types like char, short, long, longlong Nim supports these types when we put a c letter in front of the name like clong. Again, you should consult the Nim compiler manual when you need more details, i.e. when you create bindings to external libraries.

Floating-point types

Another important numeric data type is float, for floating-point numbers. Floats are an approximation of real numbers. They can also store fractions and are most often printed in the decimal system with a decimal point, or in scientific notation with an exponent. Examples for the use of variables of the float data type are

var
  mean = 3.0 / 7.9
  x: float = 12
  y = 1.2E3

The variable mean is assigned the result of the division of two float literals — the result has again the data type float, and so the compiler can infer for the type of variable mean that same type. If we printed the result of the division, there would be a decimal point and some digits following it. For variable x we specify the float type explicitly and assign the value 12. We could use type inference if we assigned 12.0 because the compiler can recognize by the decimal point that we want a float, not an int variable. In line 3 we use scientific notation for the float literal that we assign to y, and the assigned value is 1.2 * 10^3 = 1200.0. Literal values, like 2E3, are also valid float literals — the value would be 2000.0. But literals with a decimal point and no digits before or after the point — 1. or .2 — are not valid in Nim.

In the current Nim implementation, float variables always occupy 64 bits. Nim has also the data type float64, which is currently identical to plain float, and float32 which can store only smaller numbers and has less precision.[19] Floats can store values up to a magnitude of approximately 1E308 with a positive or negative sign, and floats have a typical precision of 16 digits. That is, when you do a division of two arbitrary floats and print the result, you will get up to 16 valid digits. If you tried to print more than 16 significant digits, then the additional decimal places would be just some form of random garbage. Note: The number of significant digits of a floating-point number is the total number of digits before and after the decimal point, but possible leading zero digits would not be counted. The reason that leading zeros are not significant is just that in the ordinary notation of numbers, we always assume that there is just nothing before the first non-zero digit. For our car odometer, 001234.5 km is identical to 1234.5 km. And if we give our body size as 1.80 m or 180 cm makes no difference, both values have 3 significant digits.

Generally, we use floats whenever integers are not sufficient for some reason. For example, when we have to do complicated mathematical operations which include fractional operands like Pi, or when we have to do divisions and need the exact fractional value.

The float, float32, and float64 data types provide the +, -, *, and / operators for addition, subtraction, multiplication, and division. Unlike for the int types, for the float types, we never get overflow or underflow errors, and also no error for a division by zero. But the result of an operation of two float operands can be a special value, like system.Inf, system.NegInf or system.NaN. The first two indicate an over- or underflow, and NaN (Not a Number) indicates that the result of an operation is not a valid number at all, for example, the result of a division by zero or the result of calculating the square root of a negative number. This behavior is sometimes called saturated arithmetic. When a variable has one of these special values, and we apply further math operations, then this value is kept. So we can detect at the end of a longer mathematical calculation if something went wrong — we have not to check after every single operation.[20] An interesting property of floating-point numbers is, that when we test two variables of float type for equality, and one has the value NaN, then the test is always false. That is, the test a == NaN is always false. When we forget this fact, we may initialize a float variable to the value NaN and later test with if a == NaN: to check if we have already assigned a value, which is not what we really had in mind, as that test has always a negative result. The actual test for the value NaN is a == a, which is only false when a has the value NaN, or we may use math.isNaN(). More useful constants and functions for the float data types can be found in the std/fenv module, and functions working with floats like the trigonometric ones are available from the std/math module.[21]

For floats, we have the operators +, -, *, and / for addition, subtraction, multiplication and division. For powers with integral exponents, you can use the ^ operator, but you have to import it from the std/math module. The expression x ^ 3 is the same as x * x * x. The math module contains many more functions like sin() or cos(), sqrt() and pow(). The function name sqrt() is short for square-root, and pow() stands for power, so pow(x, y) is x to the power of y when both operands have type float. For performance-critical code you should always keep in mind that pow() is an actual function call, maybe a call of a dynamic library that can not be inlined, so a call of pow(x, 2) may be a lot slower than a plain x * x. And even when using the ^ operator as in x ^ 3 we should be a bit critical. But of course, we always hope that the compiler will optimize all that for us.

The operators +, -, *, and / can be used also when one operand is a float variable and the other operand is an integer literal. In that case, the compiler knows that we really intend to do a float operation and converts the integer literal automatically to the float type. But when one operand is a float variable and the other is an int variable, then an explicit type conversion is necessary, like in float(myIntVal) * myFloatVal. For the type conversion, we use the desired type like a function, as in float(). One explanation why in this case the int value is not automatically converted to float is, that this may mean a loss in precision, as large int64 values can not be presented exactly as a float. Well, for int32 this reason does not really apply, but still there is no automatic conversion. But indeed, as Nim is used as a systems programming language, it seems to be a good decision to need explicit conversions in this case, as it makes it more clear what really is intended. And generally, we should try to avoid using a lot of operations with mixed types, as that may make type conversions necessary, which may cost performance. If we really do not care, we may import the module std/lenientOps, which defines the arithmetic operations for mixed operands.

Floating-point literals have the float data type by default, but as for integer literals, we can also explicitly specify the data type: The suffixes f and f32 specify a 32-bit float type, and d and f64 specify a 64-bit type. We can separate the suffix from the actual number with a ' character, but that is not required as long as there is no ambiguity. We can also specify float literals in binary, octal, or hexadecimal notation when we append one of these suffixes. In the case of hexadecimal notation, the ' is obviously needed to separate the suffix, as f and d are valid hex digits.

As for integer variables, Nim supports also the compatible types cfloat and cdouble which match the C types float and double when the C backend is enabled. For most C compilers, C float matches Nim’s float32 and C double matches Nim’s float64.

Some CPUs and C compilers also support other floating-point types beside the common float32 and float64 types. Intel x86 compatible CPUs generally support float80 math operations, and the GCC C compiler may support float128. But these types are not yet supported by the Nim compiler of A. Rumpf. But there may exist external packages which support these types by calling C functions when the C backend is used.

Two important properties of floats are that not all numbers can be represented exactly and that math operations are not absolutely accurate. Recall that in our decimal system, some fractions like 1/2 can be represented exactly as 0.5 in decimal notation, while others like 1/3 can be only approximated as 0.3333…​ Like all data, floats are stored internally in binary form following the IEEE Standard for Floating-Point Arithmetic (IEEE 754). In that format some values, e.g. the value 0.1, can not be represented exactly. As a result, some simple arithmetic operations, executed on the computer, will give us not exactly the result that we may expect. As we should really remember this important fact, we will investigate this behavior with a small example program where we divide a few small integer numbers after conversion to float by another to float converted integer j and sum the result j times:[22]

for i in 1 .. 10:
  echo "--"
  for j in 2 .. 9:
    let a = i.float / j.float
    var sum: float
    for k in 1 .. j:
      sum += a
    echo sum

which generates this output:

--
1.0
1.0
1.0
1.0
0.9999999999999999
0.9999999999999998
1.0
1.0
--
2.0 # for all iterations!
--
3.0 # for all iterations!
--
4.0
4.0
4.0
4.0
4.0
3.999999999999999
4.0
4.000000000000001
--
5.0
5.0
5.0
5.0
5.0
5.0
5.0
4.999999999999999
--
6.0
6.0
6.0
6.0
6.0
5.999999999999999
6.0
6.0
--
7.0
7.0
7.0
7.0
7.000000000000001
7.0
7.0
7.0
--
8.0
8.0
8.0
8.0
7.999999999999999
7.999999999999998
8.0
8.000000000000002
--
9.0 # for all iterations!
--
10.0
10.0
10.0
10.0
10.0
10.0
10.0
9.999999999999998

The echo() procedure prints up to 16 significant digits for a float value, and so the accumulated tiny arithmetic errors become visible. After our remarks above, that should be not surprising anymore, the general solution is to round results to less than 16 decimal digits before printing. Various ways to do that will be shown later in the book. A related issue of float arithmetic is caused by scaling and extinction. When we add numbers with very different magnitudes, the result may be just the value of the largest number, as in echo 1.0 == 1.0 + 1e-16, which prints true. The tiny summand is just too small to actually change the result, that is as when you switch on a torch on a sunny day, it will not really become brighter. Maybe more surprising is, that calling echo() with some simple float literals will print a different value, like echo 66.04 which gives 66.04000000000001 for Nim v1.6, while with Python3 we get 66.04 exactly. But indeed that is only surprising for people who do not understand well what a statement like echo 66.04 really does: We know already, that the value 66.04 is converted by the compiler to an internal binary representation, and then converted back to a decimal string when we run the program. So it is not that surprising that in this process some tiny inaccuracies can accumulate. Actually, it may be possible to get exact 16 digits precision when a very smart conversion routine like the Ryu or dragon box algorithm is used. We may still wonder why Python just seems to get it right — well there are rumors that Python is cheating, maybe some past-processing to guess the string that the user may like.

From the discussions above, it should be clear that testing two floats for equality is regularly problematic. Instead of just testing for equality, we may better define a small []epsilon value like eps = 1e-14 and then write (a - b).abs < eps. Seems to be not bad, and can be seen frequently and often works, but not always. Imagine you write a program that processes chemical elements, and you work with atomic mass and radii. So maybe the result with the above test is that all the atoms of the periodic table have equal mass and equal size, at least when you should use the SI system with meter and kilogram as base units. So an equality test like

const eps = 1e-16 # an arbitrary relative precision
if (a == 0 and b == 0) or (a - b).abs / (a.abs + b.abs) < eps: # avoid div by zero

if (a - b).abs / (a.abs + b.abs + 1e-32) < eps: # a similar check, avoiding also a div by zero

may be a better solution in the general case. Whenever you need a general equality test, you should think about the problem and do some tests — the code above is just an untested possible example.

At the end of this section, some remarks about the performance of float data types compared to plain ints: On modern hardware like the popular x86 systems for the basic operations performance of floats and ints is very similar, addition, subtraction, and even multiplication is basically done in only one clock cycle, and division may be a bit slower. Even operations like sqrt() which have been regarded as slow in the past, are now close to a plain addition on modern hardware. As the CPU does its float arithmetic internally with 64 or even with 80 bits, float32 is not faster than float64, as long as the operations are not memory bound, that is large data sets are processed so that it is an advantage when the data types are smaller so that more of it fits into the cache. For tiny microcontrollers and embedded devices, things are very different, as these devices typically have no floating-point units. So the compiler has to emulate all the float arithmetic, maybe by the use of libraries. This is very slow and produces large executables. So when writing software for modern desktop PCs, there is no reason to try to avoid float math, when solving the problem with float is easier. When the data extends a very wide range, e.g. from nm to millions of km, or when operations like square root or trigonometric functions are needed, then there is typically no way and reason to avoid float. In the case that floats or ints may work both, it is generally a good strategy to try to use ints on the first try. Ints may still provide better performance for SIMD, threading, and parallel processing, as ints may avoid the expensive saving of floating-point CPU registers. For restricted hardware, we should better try to avoid float math. But all this is a difficult topic, and this advise can give you only some simple recommendations, which may be wrong for a concrete case. So finally you have to decide for yourself, and as always it is a good idea to do some performance tests.

References:

Distinct types

Before we continue with subrange types, we should introduce the distinct types. In the real world, we have a lot of quantities for which the set of meaningful math operations is restricted and which should not be mixed with quantities of other types. For example, we may have the quantities time and distance measured in seconds and meters and mapped to the float or int data type. While adding seconds and adding meters is a valid operation, adding seconds to meters makes no sense and would be a program bug if it should occur in the program code. But again, dividing a distance by a time period resulting in the average speed would be a valid operation. Nim provides the distinct keyword, which allows us to define new data types that are based on existing types, but that are not compatible with them or with other distinct types. And the new defined distinct types have no predefined operations, we have to define all desired operations ourselves.

type
  Time = distinct float # in seconds
  Distance = distinct float # in meters

var t: time = 0.2 # not allowed
var t: Time = Time(0.2)

For distinct types, we have to define all the allowed operations ourselves. We can convert distinct types to the base types and then use operations of the base type, or we can borrow operations from the base type by use of the {.borrow.} pragma. Using distinct types can be complicated when the new type should support many operations, but it can make our code safer. For some data types with a very limited set of operations, distinct types can be used easily. Distinct types are explained in detail in the Nim manual, we may explain them in more detail in later sections. For now, it is enough that we know about their existence.

Subrange types

Sometimes it makes sense to limit the range of numeric variables to only a sub-range. For this, Nim uses the range keyword with this notation: range[LowVal .. HighVal]. Values of this type can never be smaller than LowVal or larger than HighVal. For Nim v1.6 we can define range types also by leaving out the range[], that is, by just two constants separated by ...

type
  Year = range[2020 .. 2023] # software update required at least for 2024!
  Month = range[1 .. 12]
  Day = 1 .. 31 # same as range[1 .. 31]

var a: int = 0
var d: Day = 1 # OK
d = 0 # compile-time error
d = a # run time test and error
echo d

For the above example, the base type of the defined ranges is int, so the ranges are compatible with the predefined int type, we can assign values of int type to our range types and vice versa. In our example, the size of the range types is the size of the int base type, but of course, we could use other base types, like type Weekday = 1.int8 .. 7.int8. If we try to assign to a range type a value that falls not into the allowed range, then we get a compile-time or run-time range error. This can help us to prevent or to discover errors in our programs. Note that whenever we use range types, the compiler may have to add additional checks to ensure that variables are always restricted to the specified range. This check is active in debug mode and also when we compile with option -d:release, and is only ignored when we compile with -d:danger or explicitly disable range checks. So using a lot of range types may increase code size and decrease performance. For the example above, the line with the assignment d = a generates a runtime check. An important and often used range type is the data type Natural, defined as range[0 .. int.high]. That type is compatible with the int type and does not wrap around as uint would do. It is regularly used as the type for procedure parameters when the arguments have to be not negative. In the proc body we sometimes copy arguments of natural type to an ordinary integer — that way we can ensure a none negative start value and can avoid many range checks in the procedure body.

We can also declare sub-range types with float base types like type Probability = range[0.0 .. 1.0].

Note that we can still mix different sub-range types:

var d: Day = 13
var m: Month = 3
d = d + m

Such an operation is generally a bug, to prevent it we can put the distinct keyword in front of our ranges. But then again we have to define the allowed operations ourselves or borrow them from the base type.

Enumeration types

Enumeration types are shortened as enum in Nim. While enums in C are nothing more than integers with some special syntax for creation, Nim’s enums are more complex.

In Nim, enums can be used whenever some form of symbols are needed like the colors red, yellow, and green of a traffic light, or the directions north, south, east, and west for a map or a game.

Most of the time, we declare an enum type and the corresponding values by simple listing them like

type
  TrafficLight = enum
    red, yellow, green

We can use variables of type TrafficLight then like

var tl: TrafficLight
tl = green
if tl == red:
  tl = ...

Enums support assignment, plain tests for (in)-equality and for smaller or greater. Additionally, the functions succ() and pred() are defined for enums to get the successor or predecessor of an enum, ord() or int() deliver the corresponding integer number and the $ operator can be used to get the name of an enum. We can also iterate over enums, so we can print all the colors of our TrafficLight by

for el in TrafficLight:
  echo el.ord, ' ', $el

Ordinary enums start at 0 and use continuous numbers for the internal numeric value so that enums can be used as array indices.[23]

type
  A = array[TrafficLight, string]

var a: A
a[red] = "Rot"
echo a[red]

But we can also assign custom numbers like

type
  TrafficLigth = enum
    red = -1, yellow = 3, green = 8

We should avoid that, as these "enums with holes" generate some problems for the compiler and may later be deprecated. For example, array indexing or iterating is obviously not possible for enums with holes.

It is also possible to set the string that the stringify operator $ returns, like in

type
  TrafficLigth = enum
    red = "Stop"
    yellow = (2, "Caution")
    green = ("Go")

Here the assigned numerical values should be 0, 2, and 3. Currently, the enum’s numerical values must always be specified in ascending order.

When we have many enums in a program, then name conflicts may occur, for example, we may have an additional enum type named BaseColor, which also has red and green members. For that case, the {.pure.} pragma exists:

type
  BaseColor {.pure.} = enum
    red, green, blue

With the pure pragma applied, we can use the full qualified enum name when necessary, like BaseColor.red. But we can still use unqualified names like blue when there is no name conflict.

Boolean types

Boolean types are used to store the result of logical operations. The type is called bool in Nim and can store only two values, false and true. Although we have only two distinct states for a boolean variable and so one single bit would suffice to store a bool, generally, a whole byte (8 bits) is used for storing a boolean variable. Most other programming languages, including C, do the same. The reason is that most CPUs can not access single bits in the RAM — the smallest entity that can be directly accessed in RAM is a byte. The default initial state of a boolean variable is false, corresponding to a byte with all bits cleared.

var
  age = 17
  adult: bool = age > 17
  iLikeNim = true
  iLikeOtherLangaugeBetter = false.

In line three, we assign to the variable adult the result of a logical comparison. The next two lines assign the boolean constants true and false to the variables, with their type bool inferred.

Variables of type bool support the operators not, and, or and xor. Not inverts the logical value, a and b is only true when both values are true, and false otherwise. And a or b is true when at least one of the values is true, and only false when both values are false. Xor is not used that often. It is called exclusive or, a xor b is false when both values have the same logical state, that is when both are true, or both are false. When the values are not the same, then the result of the xor operator is true. The xor operator makes more sense for bit operations, which we will learn later — for the boolean type, a xor b is identical to a != b.

When using conditional execution, some people like to write expressions like if myBoolExp == false:, which is identical to if not myBoolExp:. Well, they may do, but please never write if myBoolExp == true:, that looks really too stupid.

Sometimes it is useful to know that false is mapped to the int value 0, and true to the int value 1. That is similar to the C language, but C has no real boolean type, instead, the numerical value 0 is interpreted as false in conditional expressions, and all none zero values are interpreted as true.

var a: int = 0
if cond:
  a = 7

a = 7 * cond.int

The effect of the last line is identical to the if statement above. In very, very rare cases, working with the actual int value of boolean variables may make sense, but generally, we should avoid that. Later in the book, there is a section about branchless code where we will present a proc that actually may get faster by using such a trick.

Characters

The data type for single characters is called char in Nim. A variable of type char has 8 bits and can store single characters. Indeed, it stores 8-bit integers which are mapped to characters. The mapping is described by the ASCII table. For example, the integer value 65 in decimal is mapped to the character A. When we use single character literals, then we have to enclose the letter in single quotes. As only 8 bits are used to store characters, we only have 256 different values, including upper and lower case letters, punctuation characters, and some characters with a special meaning like a newline character to move the cursor in the terminal to the next line, or a backspace character to move the cursor one position backward. Single characters are not used that often, since we generally group them in sequences called strings to build text.

The initial ASCII table contains only the characters with numbers 0 up to 127, here is an overview generated with the small program listed in the appendix:

Visible ASCII Characters

      +0   +1   +2   +3   +4   +5   +6   +7   +8   +9  +10  +11  +12  +13  +14  +15
  0
 16
 32        !    "    #    $    %    &    '    (    )    *    +    ,    -    .    /
 48   0    1    2    3    4    5    6    7    8    9    :    ;    <    =    >    ?
 64   @    A    B    C    D    E    F    G    H    I    J    K    L    M    N    O
 80   P    Q    R    S    T    U    V    W    X    Y    Z    [    \    ]    ^    _
 96   `    a    b    c    d    e    f    g    h    i    j    k    l    m    n    o
112   p    q    r    s    t    u    v    w    x    y    z    {    |    }    ~

The position in the table is the sum of the number on the left and the number on the top, i.e, character A has position 64+1=65, which is the value that the Nim standard function ord('A') or int('A') would return. The characters with a decimal value less than 32 can not be printed and are called control characters, like linefeed, carriage return, backspace, audible beep, and such. Character 127 is also not printable and is called DEL. An important property of this table is the fact that decimal digits and upper- and lower-case letters form contiguous blocks. So to test for example, if a character is an uppercase letter, we can use this simple condition: c >= 'A' and c <= 'Z'.

Characters with ord() > 127 are so-called umlauts, exotic characters of other languages, and some special characters. But these characters may be different on different computers, as the characters depend on the active code page, which maps positions to the actual character, and there are multiple code pages. When we need more than the plain ASCII characters, then we use strings in Nim, which display many more glyphs by using UTF-8 encoding.

The control characters with a decimal value less than 32 can not be typed on the keyboard directly, and for some characters with a decimal value greater than 126, it can be difficult to enter them on some keyboards. For these characters, as well as for all other characters, escape sequences can be used. Escape sequences start with the backslash character, and the following characters are interpreted in a special way: The backslash can follow a numeric value in decimal or hexadecimal encoding, or a letter, which is interpreted in a special way. We mentioned already that the character 'A' is mapped to the decimal value 65, which is its position in the ASCII table. So instead of 'A', we could use the escape sequence '\65' for this character. Or, as decimal 65 is 41 in hexadecimal notation (4 * 16^1 + 1 * 16^0) we can use '\x41' where the x indicates that the following digits are hexadecimal. For common, often used control characters it is not easy to remember their numeric value, so another notation with a letter following the backslash can be used. For the important newline character, we can use the decimal numeric value '\10', the hexadecimal value '\xA', or the symbolic form '\n'. Here, the letter n stands for newline.

We can regard the backslash character, which introduces escape sequences, as a special hinting symbol for the compiler: Caution, the following characters must be interpreted in a special way.

It is important that you understand that all these escape sequences are only a way to help the programmer to enter these invisible control characters — the compiler replaces the control sequences immediately with the correct 8-bit value from the ASCII table, so in the final compiled executable '\65' or '\n' are both only a plain 8-bit integer value:

var a, b: char
a = 'A'
b = '\65'
echo a, ord(a), b, ord(b) # if you don't know the output, read again this section and run this code.

The following table lists a few important control characters:

Decimal Hexadecimal Symbolic Meaning

10

xA

\n, \l

newline or linefeed — move cursor one position down

12

xC

\f

formfeed

9

x9

\t

tabulator

11

xB

\v

vertical tabulator

92

x5C

\\

backslash

39

x27

\'

single-quote, apostrophe

7

x7

\a

alert, audible beep

8

x8

\b

backspace

27

x1B

\e

Escape, [ESC]

13

xD

\r, \c

return or carriage return — move the cursor at the beginning of the line

The hexadecimal numbers after the \x character can be upper or lower case and can have one or two hexadecimal digits. For symbolic control characters like '\a' for alert, the upper case variant '\A' seems to be identical currently. The single quote entered as ''' does give an error message, so you have to escape it as '\''. Unfortunately, by supporting this form of escaping it becomes impossible to enter a backslash character directly, so we have to escape the backslash character as '\\' to print a single backslash.

For Nim, the most important control character is '\n', which is used to start the output in a terminal window at the beginning of a new line. But '\n' is generally not used as a single character but embedded in strings, that is, sequences of characters. We will learn more about strings soon. Note that the echo() function inserts a newline character automatically after each printed line, but the write() function does not:

echo 'N', 'i', 'm'
stdout.write 'N', 'i', 'm', '\n'

What may be a bit confusing is the fact that we use the backslash character as an escape symbol, and at the same time, the above table has an entry '\e', which is also called [ESC]. These '\e' control character with decimal value 27 is fully unrelated to the backslash character that we use to type in control characters. [ESC] is a different special character to start control sequences, it was used in the past to send special commands to printers or modems, and can be used to control font style or colors in terminal windows.

Nim’s control characters should, with few exceptions, be identical to the control characters of the C language, so you may also consult C literature for more details.

Ordinal types

In Nim integers, enumerations, characters, and the boolean types are ordinal types. Ordinal types are countable and ordered, and for each of these types, a lowest and largest member exists. The integer ordinal types support the inc() and dec() operations to get the next larger or next smaller value, and the other ordinal types use succ() and pred() for this operation. These operations can produce overflow or underflow like errors if applied to the largest or smallest value. The function ord() can be used on ordinal types to get the corresponding integer value. Note that unsigned integers are currently not called ordinal types in Nim and that these unsigned types wrap around instead of generating overflow and underflow errors.

Sets

From mathematics, we know that sets are some form of unordered collection for which we can test membership (x is included in mySet) and we can perform general set operations like building the union of multiple sets. In Nim, we can have sets of all the ordinal types and the unsigned integer types, but due to memory restrictions, integer types larger than two bytes can not be used as set base types. All elements in a set must have the same base type. A set can be empty, or it can contain one or multiple elements. For a specific element, it can be contained in a given set, or it can be not contained, but it can be never contained multiple times. One very basic set operation is to test if an element is contained in a set or is not contained in it. Sets are unordered data types, that is sets containing the same elements are always equal, it does not matter in which sequence we added the elements. Important set operations are building the union and building the difference of two sets with the same base type: The union of set a and set b is a set that contains all the elements that are contained in set a or in set b (or in both). The intersection of set a and set b is a set that contains only elements that are contained in set a and in set b.

The mathematical concept of sets maps well to words and bits of computers, as most CPU’s have instructions to set and clear single bits and to test if a bit is set or unset. And CPUs can do and, or and xor operations which corresponds to the union and intersection operation in mathematical sets.

Nim supports sets with base type bool, enum, char, int8, uint8, int16, and uint16. Note that we need a bit in the computer memory for each member of the base type. The types char, int8, and uint8 are 8-bit types and can have 2^8 = 256 distinct values, so we require 256 bits in the computer memory to represent such a set. That would be 32 bytes or four 64-bit words. To represent a set of the base type uint16 or int16, we need already 2^16 bits, that is 2^13 bytes or 2^10 words on a 64-bit CPU. So it becomes clear that supporting base types with more than 16 bit makes not much sense.

While testing, if an element is included or is not included in a set with the in or notin operators, is always a very fast operation, other operations like building the intersection or union and set comparison operations may not be that fast when we use the int16 or uint16 base types, as for these operations the whole set, that is 2^10 words on a 64-bit CPU, has to be processed.

We will start our explanation with sets with character base type, as these sets are very easy to understand and at the same time very useful. Let us assume that we have a variable of character type, and we want to test if that variable is alphanumeric, that is if it is a lower or upper case letter or a digit. A traditional test would be (x >= a and x <=+z) or (x +>= A and x <= Z) or (x >= 0 and x <= 9). Using Nim’s set notation, we can write that in a simpler form:

const
  AlphaNum: set[char] = {'a' .. 'z', 'A' .. 'Z', '0' .. '9'}

var x: char = 's'
echo x in AlphaNum

Here we defined a constant of set[char] type which contains lower and upper case letters and the decimal digits. We used the range notation to save us a lot of typing ({'a', 'b', 'c', …​}). It works in this case only, as we know that all the lowercase letters, the upper case letters, and the decimal digits built an uninterrupted range in the ASCII table.

With that definition, we can use a simple test with the in keyword. This test is equivalent to the procedure call AlphaNum.contains(x). In compiled languages (most) set operations are generally very fast as they map well to CPU instructions.

Older languages like C do not have a dedicated set data type, but as sets are so useful and efficient, C emulates these operations by using bit-wise and and or operations in conjunction with bit shifts.

Two important operations for sets are building the union and the intersection:

const
  AlphaNum: set[char] = {'a' .. 'z', 'A' .. 'Z', '0' .. '9'}
  MathOp = {'+', '-', '*', '/'} # set[char]

  ANMO = AlphaNum + MathOp # union
  Empty = AlphaNum * MathOp # intersection

The constant ANMO would now contain all the characters from AlphaNum and from MathOp, that is letters, digits, and math operators. The constant Empty would get all the characters that are at the same time contained in set AlphaNum and in set MathOp. As there is not a single common character, the set Empty is indeed empty. Remembering the two operators + and * for union and intersection is not easy. For the intersection operator * it may help when we imagine the set members as bits, and we assume that we multiply the bits of both operands bitwise, that is we multiply the set or unset bits at corresponding positions each. The resulting bit pattern would get set bits only for positions where both arguments have set bits.

We can use the functions incl() and excl() to add or remove single set members:

var s: set[char]
s = {} # empty set
s = {'a' .. 'd', '_'}
s.excl('d')
s.incl('?')

The result is a set with letters a, b, c and the characters _ and ?. Note that calling incl() has no effect when the value is already included in the set, and calling excl() has no effect when the value is not contained in the set at all.

Another operation is the difference of two sets — a - b is a set that contains only the elements of a which are not contained in b. In Nim, there is currently no operator for the complement or the symmetric difference of sets available. We can produce a set complement by using a fully filled set and then removing the elements of which we want the complement. For a character set that would look like {'\0'..'\255'} - s, where s is the set to complement. And the symmetric difference of set a and set b can be generated by the operation (a+b) - (a*b) or by (a-b) + (b-a).

As the not operator binds more tightly than the in operator, we have to use brackets for the inverted membership test like not(x in a) or we can use the notin operator and write x notin a. We can test for equality of sets a and b like a == b and for subset relation a < b or a <= b. a <= b indicates that b contains at least all members of a, and a < b that b contains all members of a and at least one more element.

Finally, we can use the function card() to get the cardinality of a set variable, that is the number of contained members.

We should also mention that we can have character sets that are restricted to a range of characters:

type
  CharRange = set['a' .. 'f']

# var y: CharRange = {'x'} #invalid

var y: CharRange = {'b', 'd'}
echo 'c' in y

In the code above, the compiler detects the first assignment to variable y as invalid.

Sets of numbers work in principle in the same way as sets of characters. One problem is that in Nim integer numbers are generally 4 or 8 bytes large, but sets can contain only numbers with 1 or 2-byte size. So we have to specify the type of set members explicitly:

type
  ChessPos = set[0'i8 .. 63'i8]

var baseLine: ChessPos = {0.int8 .. 7.int8}
var p: int8
echo p in baseLine

In the code above, we defined a set type that can contain int8 numbers in the range 0 to 63.

We can use also another notation for numeric sets when we define an explicit range type like in

type
  ChessSquare = range[0 .. 63]
  ChessSquares = set[ChessSquare]

const baseLine = {0.ChessSquare .. 7.ChessSquare}
# or
const baseLineExplicit: ChessSquares = {0.ChessSquare .. 7.ChessSquare}
assert baseLine == baseLineExplicit

What may be a bit surprising is the fact that Nim’s sets work also for negative numbers:

type
  XPos = set[-3'i8 .. +2'i8]

var xp: XPos = {-3.int8 .. 1.int8}
var pp: int8 = -1

Enum sets are also very useful and can be used to represent multiple boolean properties in a single set variable instead of using multiple boolean variables for this purpose:

type
  CompLangFlags = enum
    compiled, interpreted, hasGC, isOpenSource, isSelfHosted
   CompLangProp = set[CompLangFlags]

const NimProp:  CompLangProp = {compiled, hasGC, isOpenSource, isSelfHosted}

Enum sets can be used to interact with functions of C libraries, where for flag variables often or’ed ints are used. For example, for the gintro C bindings there is this definition:

type
  DialogFlag* {.size: sizeof(cint), pure.} = enum
    modal = 0
    destroyWithParent = 1
    useHeaderBar = 2

  DialogFlags* {.size: sizeof(cint).} = set[DialogFlag]

Here, the {.size.} pragma is used to ensure that the byte size of that set type matches the size of integers in C languages.

When we define a set of enums in this way to generate bindings to C libraries, then we have to ensure that the enum values start with zero, otherwise, Nim’s definition will not match with the C definition. For example, in the gdk.nim module we have

type
  AxisFlag* {.size: sizeof(cint), pure.} = enum
    ignoreThisDummyValue = 0
    x = 1
    y = 2
    pressure = 3
    xtilt = 4
    ytilt = 5
    wheel = 6
    distance = 7
    rotation = 8
    slider = 9

  AxisFlags* {.size: sizeof(cint).} = set[AxisFlag]

The first enum with ordinal value zero was automatically added by the bindings generator script to ensure type matching. Nim’s devs sometimes recommend to use plain (distinct) integer constants for C enums. That may be easier, but integer constants provide no namespaces, so names may be aFlagWheel instead of AxisFlag.wheel or plain wheel when there is no name conflict for pure enums. And with integer constants, we have to combine flags by or operation like (aFlagWheel or aFlagSlider) instead of clean {AxisFlag.wheel, slider}.

Can we print sets easily? As sets are an unordered type, it is not fully trivial, but we can iterate over the full base type and check if the element is contained in our set like

var s: set[char] = {'d' .. 'f', '!'}

for c in 0.char .. 255.char:
  if c in s:
    stdout.write(c, ' ')
echo ' '
! d e f

We will learn how the for loop work soon. Note that the sequence in which the set members are printed is determined by our query loop, not by the set content itself, as sets are unordered types.

Strings

The string data type is a sequence of characters. It is used whenever a textual input or output operation is performed. Usually, it is a sequence of ASCII-only characters, but multiple characters in the string can be interpreted as so-called utf-8 Unicode characters, that allow displaying nearly unlimited symbols as long as all the needed fonts are installed on your computer, and you manage to enter them — Unicode characters may not be accessible by a simple keystroke. For now, we will only use ASCII characters, as they are simpler and work everywhere. String literals must be enclosed in double-quotes. Nim’s strings are similar to the Nim seq data types: both are homogeneous variable-size containers. That means that a string or a seq expands automatically when you append or insert characters or other strings. Nim’s seq data type is discussed later in the book in some detail. Don’t confuse short strings with only one character with single characters: A string is a non-trivial entity with an internal state like data buffer (the actually contained characters), length and storage capacity, while a variable of char type is nothing more than a single byte interpreted in a special way. So a string like "x" is fully different from 'x'.

var
  str: string = "Hello"
  name: string
echo "Please tell me your name"
name = readLine(stdin)
add(str, ' ')
echo str, name

In the above example code, we declare a variable called str and assign it the initial literal value "Hello". We use the echo() procedure to ask the user for his name, and use the readLine() procedure to read the user input from the terminal. To show how we can add characters to existing string variables, we call the add() procedure to append a space character to our str variable, and finally, call the echo() procedure to print the hello message and the name to the screen. Note that the echo() procedure automatically terminates each output operation with a jump to the next line. If you want an output operation without a newline, you can use the similar write() procedure. But write() needs an additional first parameter, for which we use the special variable stdout when we want to write to the terminal window.

So we could substitute the last two lines of the above code by

write(stdout, str)
write(stdout, ' ')
echo name

The Nim standard library provides a lot of functions for creating and modifying strings, most of these functions are collected in the system and in the strutils module. The most important procedures for strings are len() and high(). The len() procedure returns the length of a string, that is, the number of ASCII characters or bytes that the string currently contains. The empty string "" has length zero. Note that the plain len() function returns the number of 8-bit characters, not the number of Unicode glyphs when the string should be interpreted as Unicode text. To determine the number of glyphs of Unicode strings, you should use some of the unicode modules. The high() function is very similar to the len() function, it returns the index of the last character in the string. For each string s high(s) == len(s) -1, so high("") is -1. Remember that Nim supports the method call syntax, so we can also write s.len instead of len(s).

The most important operators for strings are the subscript operator [] which allows access to individual characters of strings, and the .. slice operator, which allows access to sub-strings. The first character in a string has always the index zero. For concatenation of string literals or string variables, Nim uses the & operator.

var s = "We hate " & "Nim?"
s[3 .. 6] = "like"
s[s.high] = '!'

In the example above, we define the string variable s by the use of two literal strings to show the use of the concatenation operator. In line two we use the slice operator to replace the sub-string "hate", that is, the characters with index position 3 up to 6, with the string literal "like". In this case, the replacement has exactly that many characters as the text to replace, but that is not necessary: We can replace sub-strings with longer or shorter strings, which includes the empty string "" to delete a text area. In the last line of the above example, we use the subscript operator [] to replace the single character '?' at the end of our string with an exclamation mark. For subscript and slice operators, Nim supports also a special notation that indicates indexing from the end of the string. Python and Ruby use negative integers for this purpose, while Nim uses the ^ character. So [^1] is the last character, [^2] the one before the last. So we could have written s[^1] = '!' for the last line of our code fragment above. The reason that Nim does not use negative integers for this purpose is that Nim arrays don’t have to start at index zero, but can start with an arbitrary index including negative indices, so for negative indices it may not always be clear if a regular index or a position from the end of the string is desired. The term s[^x] is equivalent to s[s.len - x]. We will learn some more details about the slice operator in a later section when we have introduced arrays and sequences.

Another important operator for strings is the "toString" or stringify operator $. It can be applied to variables of nearly all data types and returns its string representation, which can then be printed. Some procedures like echo() apply this operator on its arguments automatically. When we define our own data types, then it can make some sense to define the $ for them, in case we need a textual representation of our data — maybe only for debugging purposes. Note that applying the $ operator on a string directly has no effect and is ignored, as the result would not change.

Strings can contain all characters of the char data type, including the control characters. The most essential control character for strings is the newline character '\n', which is used at the end or sometimes also in the middle of strings to start a new line. For strings, Nim also supports the virtual character "\p" to encode an OS-dependent line break. When compiled for Windows, "\p" is automatically converted to "\r\n", and to a plain '\n' on Linux. Note that "\p" can be used in strings, but not as a single character, as it is two bytes on Windows. "\p" is only needed to support very old Windows versions or potentially another exotic operating system, as modern Windows recognizes plain '\n' well.

As strings support utf-8 Unicode, an escape sequence starting with "\u" is supported to insert Unicode code points. The "\u" follows exactly 4 hexadecimal digits or an arbitrary number of hex digits enclosed in curly braces {}.

As string literals are enclosed in quotation marks, it follows that strings can not directly contain this character, we have to escape it as in "\"Hello\", she said".

Perhaps we should mention that Nim strings use copy semantics for assignment. As we have not yet introduced references or pointers, copy semantics is what you should expect — strings behave just like all the other simple data types we used before like integer or float numbers or enums and characters:

var
  s1: string
  s2: string
s1 = "Nim"
s2 = s1
s1.add(" is easy!")
echo s1 & "\n" & "s2"

The output is

Nim is easy!
Nim

The assignment s2 = s1 creates a copy of s1, so the subsequent add() operation does only modify s1, but not s2. Probably not surprising for you, but other programming languages may behave differently, i.e. the assignment may not copy the textual content, but create only a reference to the first string so that modifying one of them also affects the other. We will learn more about the concept of references when we introduce the object data type.

Entering Unicode Characters

UTF-8 is a variable-width character encoding. To cite the introducing section from https://en.wikipedia.org/wiki/UTF-8:

UTF-8 is capable of encoding all 1,112,064[nb 1] valid character code points in Unicode using one to four one-byte (8-bit) code units. Code points with lower numerical values, which tend to occur more frequently, are encoded using fewer bytes. It was designed for backward compatibility with ASCII: the first 128 characters of Unicode, which correspond one-to-one with ASCII, are encoded using a single byte with the same binary value as ASCII, so that valid ASCII text is valid UTF-8-encoded Unicode as well. Since ASCII bytes do not occur when encoding non-ASCII code points into UTF-8, UTF-8 is safe to use within most programming and document languages that interpret certain ASCII characters in a special way, such as "/" (slash) in filenames, "\" (backslash) in escape sequences, and "%" in printf.

In Nim we have four ways to enter Unicode characters: As hexadecimal digits following the "\x", as Unicode code point following the "\u" or we can type the Unicode sequence directly on our keyboard, as one single keystroke when our keyboard layout supports that, or as a special OS-dependent sequence of keystrokes:

echo "\xe2\x99\x9A \xe2\x99\x94"
echo "\u265A \u2654"
echo "\u{265A} \u{2654}" # {} is only necessary for more than 4 hex digits
echo "♚ ♔"

The code above shows three ways to print the symbol for a black and a white king of a chess game. In the first line, we typed the Unicode sequence directly as hexadecimal digits, this method is rarely used today. In the second line, we used "\u" to enter the code point directly, we get the code from https://en.wikipedia.org/wiki/List_of_Unicode_characters. And finally, we entered the glyph directly in an editor. For some Linux editors like gedit we can hold down shift and control key and then type u, release all keys and type the Unicode digits like 265a and a space. See https://en.wikipedia.org/wiki/Unicode_input for details and other operating systems.

The CString data type

In the C programming language, strings are just pointers to sequences of characters of fixed length.[24] The end of such a C string is generally marked with the character '\x0' — a null byte with all bits cleared. C functions like printf() needs these "\x0" characters to determine the end of the C string. While Nim strings are complex entities that store their current size and other properties and can grow dynamically, the character sequence of Nim strings has also a hidden terminating '\x0' character at the end to make them compatible with C strings. Nim has also the data type cstring, called "compatible string" in modern Nim, which matches the strings in C language if we compile as usual with the C backend. Cstrings are used in bindings definitions for C libraries, but as cstrings can not grow and do support only a few string operations, they are only used in rare cases in ordinary Nim source code. The Nim compiler passes automatically the zero terminated data buffer of Nim strings to C libraries whenever we call a C library, so there is no expensive type conversion involved. But the other way is much more expensive: When you have an existing cstring and need a Nim string with the same content, then a simple conversion is not possible as a Nim string is a different, more complex entity. So we have to create a Nim string and copy the content, you can use the stringify operator $ for this like in myNimStr = $myCString. Generally, string creation is an expensive operation compared to plain operations like adding two numbers, so when performance matters one should try to avoid unnecessary string creation and also unnecessary string operations. This is most important in loops, which are executed often. We will explain more about the internals of strings and why string creation and dynamically allocating memory is expensive in later sections of the book.

When we access text ranges with the slice operator or single characters with the subscript operator, we should never access indices below the currently last index, which is the index mystr.high or ^1. If we do that, we get an exception, as that index would contain undefined data or would not exist at all. We said earlier that Nim strings grow automatically if we insert or append data. But that does not mean that we can use the subscript or slice operator to access characters after the current end of the string. Such an operation would really make not much sense: Imagine we have a string var str = "Nim" and now use the subscript operator and assign a character at position 10 with str[10] = '!'. What should become the content of characters 4 .. 9? Well, maybe spaces would make some sense, but in fact, such access after the currently last valid character of the string is forbidden. You could do str.add(" !") for this purpose.

Another operation you should avoid is inserting the '\x0' null byte character somewhere in an existing Nim string. Nim stores the actual length of strings explicitly and additional terminates the end of the actual data with a '\x0' to make the string compatible with C strings and allow passing the data buffer directly to C library functions. A '\x0' character somewhere in the middle of a Nim string would generate an inconsistency, as C library functions like printf() would regard '\x0' as the string end marker, while pure Nim functions may assume still a longer string. Intermediate '\x0' bytes in strings can in very rare cases be a problem when we get the actual byte sequence from C libraries. For the same reason, a Nim string is not identical or fully compatible with s seq[char], as a seq[char] may contain multiple zero byte data, while Nim strings should not.

Escape Sequences in Strings

We learned about control characters already in the section about characters, and earlier in this section, we mentioned that …strings can also contain control characters. As the use of control characters may not be really easy to understand, we will explain their use in strings in some more detail and give a concrete example.

The most important control character for strings is the newline character, which moves the cursor in the terminal window to the beginning of the next line. The echo() procedure prints that character automatically after each output operation. Indeed, it can be important, to terminate each output operation with that character, as the output can be buffered, and writing just a string without a termination newline may not appear at once on the screen, but can be delayed. That is bad when the user is asked something and should respond, but the message is still buffered and not yet visible.

The problem with special characters like backspace or newline is that we can not enter them directly with the keyboard.[25] To solve that problem, escape sequences were introduced for most programming languages. An escape sequence is a special sequence of characters, that the compiler can discover in strings and then replace with a single special character. Whenever we want a newline in a string we type it as "\n", that is, the backslash character followed by an ordinary letter n, n for newline.

echo "\n"
echo "Hello\nHello\nHello"

The first line prints two empty lines — two because the \n generates a jump to next the line, and because echo() always adds one more jump to the next line. The second line prints three lines, which each contains the word Hello, and the cursor is moved below the last Hello, because echo() automatically adds one more newline character.

Older Windows versions used generally a sequence of two control characters to start a new line, one '\r' (carriage-return) to move to the start of the line, and one '\l' (linefeed) to move down. You may still find these two characters in old Windows text files at the end of each line. Old printers used this combination too, so it was possible to send that text files to old printers directly. Nim also has the special escape sequence "\p", which is called platform-dependent newline and maps to "\c\l" on Windows. That is, when we compile our program on Windows, then the compiler replaces "\p" in our strings with a carriage-return and a linefeed character, and when we compile on Linux, then the compiler replaces "\p" only with a newline character. But modern Windows supports '\n', so we generally can use that.

Raw Strings and multi-line Strings

In rare situations, you may want to print exactly what you have typed, so you do not want the compiler to replace a '\n' with a newline character. You can do that in two ways: You can escape the escape character, that is, you put in front of the backslash one more backslash. When you print the string "\\n" you will get a backslash and the n character in your terminal. Or you can use so-called raw strings, that is, you put the character r immediately in front of your string like

echo r"\n"
echo "\\n"

Multi-line strings are also raw strings, that is, contained escape sequences are not interpreted by the compiler, and additional multi-line strings, as the name implies, can extend over multiple lines of the source text. Multi-line text starts and ends with three quotes like in

echo """this is
three lines
of text"""

echo "this is\nthree lines\nof text"

Both echo() commands above generate exactly the same machine code!

Comments

Comments are not really a data type, but they are also important. Ordinary comments start with the hashtag character # and extend to the end of the line. The # character itself and all following characters up to the line end are ignored by the compiler. You can also start the comment with ##, then it is a documentation comment. It is also ignored by the compiler but can be processed when you use tools to generate documentation for your code. Documentation comments are only allowed at certain places, often they are inserted at the beginning of a procedure body to explain its use. There are also multi-line comments, which start with the two characters #[ and end with ]#. These forms of comments can extend over multiple lines and can be nested, that is, multi-line comments can contain again plain or multi-line comments.

# this is comment
## important note for documentation
#[ a longer
but useless comment
#]

Multi-line documentation comments also exist and can also be nested.

proc even(i: int): bool =
  ##[ This procedure
  returns true if the integer argument is
  even and false otherwise.
  ]##
  return i mod 2 == 0

You can also use the #[ comment ]# notation to insert comments everywhere in the source code at places where a white-space character is allowed, but this form of in-source comments is rarely used.

Other data types

There exist some more predefined types like the container types array and seq, which can contain multiple elements of the same type, or the tuple and the object type which can contain data of different types. Nim tuples and objects are similar to C structs, they are not so verbose as Java classes. We will learn more about all these types in later sections of the book.

Nim Source Code

You have already seen a few examples of simple Nim source code. The code is basically a plain text file consisting of ASCII characters, that is, the ordinary characters that you can type on your keyboard. Generally, Nim source code can also contain Unicode utf-8 characters, so instead of using names consisting of ASCII characters for your symbol names, you could just use single Unicode characters or sequences of Unicode characters. But typically that makes not much sense, entering Unicode is not that easy with a keyboard, and it is displayed only correctly on the screen or in the terminal when the editor or terminal supports Unicode properly and when all necessary fonts are installed. That may be the case for your local computer, but what when someone others may edit your source code?

Starting with Nim version 1.6, we got some support for Unicode operators, which may be useful for some applications. For details, please see the Nim compiler manual.

Nim currently does not allow inserting tabular characters (tabs) in your source code, so you have to do the indentation of blocks by spaces only. Typically, we use two spaces for each indentation level. Other numbers work also, but you should stick to a fixed value.

Names in Nim, as used for modules, variables, constants, procedures, user-defined types, and other symbols, may contain lower and upper case letters, digits, Unicode characters, and additional underscores. But the names are not allowed to start with digits or to start or end with an underscore, and one underscore may not follow directly after another underscore.

var
  pos2: int # OK
  leftMargin: int # OK
  next_right_margin: int # OK
  _private: int # illegal
  custom_: int # illegal
  strange__error: int # illegal

Generally, we use camel-case like leftMargin for variable names, not snake-case like left_margin.

Current Nim has the special property that names are case-insensitive and that underscores are simply ignored by the compiler. The only exception is the first letter of a name, that letter is case-sensitive. So the names leftMargin, leftmargin, and left_margin are identical for the compiler. But LeftMargin is different from all the others because it starts with a capital letter. This may sound a bit strange at first but works well in practice. One advantage is, that a library author may use snake_case in his library for names, but the users of the library can freely decide if they prefer camelCase. But still, you may think that all this generates confusion. In practice, it does not, it prevents confusion. Imagine a conventional programming language, fully case-sensitive and not ignoring underscores: In a larger program, we may then have names like nextIteration and next_Iteration or keymap and keyMap. What when both names are visible in the current scope, and we type the wrong one. The compiler may not detect it when types match, but the program may do strange things. Nim would not allow that similar-looking names, as the compiler would regard them as identical and would complain about a name redefinition.

You may ask why the first letter is case-sensitive. That is to allow for user-defined types to use capital type names and then write something like var window: Window. So we can declare a variable named window of a user-defined data type named Window. That is a common practice.

The case-insensitivity and the ignoring of underscores may not be the greatest invention of Nim, but it does not really hurt. The only exception is when we make bindings to C libraries, where leading or trailing underscores are used, which can make some renamings necessary.

The only minor disadvantage of Nim’s fuzzy names is when you use tools like grep or your editor search functionality: You could not be sure if a search for "KdTree" would give you all results, you would have to try "Kd_Tree" or "KDTree" and potentially some more variants too. For that task, Nim provides a tool called nimgrep that does a case- and style-insensitive search. And possibly your editor supports that type of search also. You can also enforce a consistent naming scheme when you call the compiler with the command line argument --styleCheck:error or --styleCheck:hint.

Languages like C use curly braces to mark blocks, while other languages like Pascal use begin/end keywords for this purpose. At the same time, blocks are generally indented by tabs or spaces to make it easier for the programmer to recognize the extent of the block. This is some redundancy, which is not always helpful — block marks and indentation ranges can contradict each other and can generate strange bugs. Like Python or Haskell, Nim does not need additional block markers; the indent level is enough to mark the block extents for the compiler and the human programmer. This style looks clean and compact and was used in pseudocode of textbooks for decades already. Some people still argue that this style is less "safe", as the behavior of the code depends on invisible whitespace. But this is a strange argumentation — the whitespace is always visible due to the fact that there are visible characters on the right. Of course, changing the indention of the last line of a block would affect the behavior of the code. But that change is well visible. And program code contains many locations where changing one character breaks it. All numeric literals would suffer from adding a digit or deleting a digit. Or the operators like ++ or += from C — the code may compile well after deleting the leading +, but it would be wrong. Computer programming is working carefully! Indeed, the use of curly braces for blocks has some advantages; e.g. many editors can highlight such blocks well, the editor may support jumping back and forth between the braces, and for really large blocks it may indeed be simpler to discover the whole block range. But the practice has shown that marking blocks with indentation only works fine; most people who have used it for some time just prefer it.

Whenever you convert source code from other programming languages to Nim, you should first ensure that the original code is fully correct indented. Some editors can ensure or fix this, or you may use external tools. If you do not care for this fact, and you convert maybe from C to Nim and remove the braces of blocks, you may get it wrong when the initial indentation was wrong.

Blocks, Scopes, Visibility, Locality, and Shadowing

Like most other programming languages, Nim has the concept of code blocks or scopes. The body of procedures, functions, iterators, and templates, as well as the body of various loop constructs or code following conditional statements, builds an indented block and creates a new scope. In this new scope, we can define variables, named constants, or types with the var, let, const and type keywords that are local to this block. These symbols are only visible in this scope, and local variables that need storage are actually created when the program executes the block and destroyed when the block is left. Well, in principle, and at least for ordinary stack-allocated value variables, for references and pointer variables, things are a bit more complicated, we will discuss that in more detail when we introduce references. Here we have used the term code block, to clearly separate them from the const, var, type, and import sections, which are a different form of indented blocks. Remember that the compiler processes our program code from the top to the bottom, so we have always to define symbols before we can actually use them. When we define an entity in a code block, and a symbol with that name was already declared before outside this block, then that symbol is shadowed, that is, the prior declaration gets temporarily invisible.

proc doSomething =
  type NumType = int
  const Seven = 7
  var a: NumType = Seven
  var b: bool = true
  if b:
    echo a, ' ', b # variables of outer scope are visible
    var a, sum: float # now outer a is shadowed
    a = 2.0
    sum = a * a + 1
    echo a, ' ', sum # local data only visible in if block

  echo a # initial int variable with value 7 become visible again

doSomething()

While we have not officially introduced procedures as units to structure our program code yet, we have put the above code this time by intent into the body of a proc called doSomething().

This way, we can guarantee that the two variables a and b defined in that proc are really stack-allocated. Actually, in real-life programs, nearly all the program code is embedded in procs. We will discuss the peculiarity of global code later. In the proc body from above the two variables a and b are local to the proc doSomething() — they are created on the stack when the procedure is called, that is, when we ask to start its execution by a statement like doSomething(). These two variables are never visible in code outside this proc, and the storage for these two variables is automatically released when the execution of that procedure ends, in this case when the last line of the proc is reached. In the body of the proc, we define also a new custom type and a named constant — just to show that it is possible. Both symbols are also local to this proc and invisible outside.

The indented block following the if b: statement is sometimes called an "if then" block or just if block — in that block we define two other variables called a and sum of float type, which are also stack-allocated. If these two variables are already allocated when the proc starts its execution, or only when the then block following the if statements is executed, is actually an implementation detail. As the variable a of float type in the if then block has the same name as the outer variable of int type, that integer variable is shadowed in the if block — the outer value gets temporarily invisible as soon as the new symbol is declared. Other symbols of outer scopes remain visible. In the if then block as well as in most other indented code blocks we could also define named constants or custom types, these would be visible only in this block. Indented code blocks can be nested — in one block we can have more indented blocks, for which all declared symbols are again local and invisible outside. The last echo() statement in our code example from above is already below the if then block, so the initial variable a of integer type becomes again visible.

Global code

In the introducing sections of the book, we have generally used program code at a global level, not embedded in a proc body. We did that for simplicity and as we had not already introduced procs. Global code is sometimes used in small scripts or for special purposes, like program initialization. But for larger programs, most of the code is typically grouped into procedures. For variables defined in global code, it is not that well-defined where they are stored, it may depend on the actual Nim compiler implementation and the compiler backend. The performance of global code can be worse than code enclosed in proc bodies, so when performance matters we should put our code in procs. One reason for the not optimal performance of global code is, that global variables are not located on the stack, but in the global BSS segment of the program, and that the backend can not optimize global code that well, e.g. global variables may not be cached in CPU registers. Note that variables that have to exist and keep their value for the whole runtime of the program, and not only for the duration of the execution of a single proc, have to be defined as global. The same holds obviously for global variables that are used from code of different procs, like the stdout and stdin variables of the system module. An alternative to the use of global variables when a variable used in a proc should keep its value between different proc calls is to attach the {.global.} pragma to a proc local variable. This way that variable is still only visible in that proc where the variable is declared, but the variable is stored in the BSS segment instead of on the stack and so its value is preserved between proc calls.

Note that structured named constants, e.g. constant strings, are stored also in the BSS segment, even when they are only defined local to a proc. So large structured constants can increase the executable size, as the BSS segment is a part of the program executable.

Whitespace, punctuation, and operators

The space character with decimal ASCII value 32 is used in Nim program code to indent code blocks and to separate different symbols from each other. Nim’s keywords are always separated with leading and trailing whitespace from other symbols, while other symbols are most often separated by punctuation and an additional optional space character. Whenever the syntax allows a space, we can also insert multiple spaces or a comment enclosed in #[ ]# in the source code. Tabulator characters are not allowed in the Nim source code, but we can use them in comments and of course in string literals. We mentioned already, that spaces can make a difference in how operators or function parameters are handled. In expressions like a+b or a + b the + operator is regarded as an infix operator, but in a + -b the minus sign is regarded as a unary operator bound to b. This way asymmetric expressions like a +b or a <b would be invalid, as the operators are interpreted as unary ones attached to b, and then there is no infix operator between the two variables. A proc call like echo(1, 2) is interpreted as a call of echo() with two integer literal arguments, while a call like echo (1, 2) with a space after the proc name is interpreted in command invocation syntax as a call with a tuple argument. While in C code it is not uncommon to always insert a space between the function name and its parameter list, we should avoid that in Nim for the described reason. We will learn more about proc calls and the tuple data type later.

Operators

Nim uses the following punctuation characters as operators:

=, +, -, *, /, <, >, @, $, ~, &, %, |, !, ?, ^, ., :, \

These symbols can be used as single entities or in combination, and we can define our own operators or redefine existing operators. All these symbols can be used as infix operators between two arguments, or as unary prefix operators, but Nim does not support unary postfix operators, so a notation like i++ as known from the C language is not possible in Nim. There exist a few combinations of these punctuation characters that have a special meaning, we will learn more about that and how we can define our own operators later in the book.

In Nim, these keywords are also used as operators:

and, or, not, xor, shl, shr, div, mod, in, notin, is, isnot, of, as, from.

Operators have different priorities, e.g. * and / have a higher priority than + and -. In most cases, the priority is just as we would expect, maybe with a few exceptions. If we are unsure, we can group terms with brackets, or consult the compiler manual for details.

Since version 1.6 Nim also allows to define and use a few Unicode operators, but these are still considered experimental. For details, you should consult the compiler manual.

Order of Execution

Global program code or code in called procs is generally executed from top to the bottom and from left to the right, as long as control structures do not enforce a different order. To demonstrate this, we use here a set of four different procs, which contain an echo() statement each, and return a numeric expression. Well, we have not yet formally introduced procedures, so if you should feel too uncomfortable with the code below, just skip this section for now and come back when you have read the section about procs:

proc a(i: int): int =
  echo "a"
  i * 2

proc b(i: int): int =
  echo "b"
  i * i

proc c(i: int): int =
  echo "c"
  i * i * i

proc d(i: int): int =
  echo "d"
  i + 1

echo a(1); echo b(1)
echo b(2) + d(c(3)) # (2 * 2) + ((3*3*3) + 1)
echo "--"
echo a(1) < 0 and b(1) > 0
echo a(1) > 0 or b(1) > 0

It should be no real surprise, that the first three echo() statements produce this output:

a
2
b
1
b
c
d
32

For the term d(c(3)) it is obvious that the inner expression c(3) has to be evaluated first before that result can be used to call proc d().

The last two lines demonstrate the so-called short-cut-evaluation for expressions with the boolean and or or operators. As the expression a() and b() is always false when a() is false, in this case, b() has not to be evaluated at all. In a similar way, as the expression a() or b() is always true when a() is true, in that case, b() has not to be evaluated at all. So in the last two lines of the above code b() is never called at all, and the output is just

a
false
a
true

Note that in Nim as in most other programming languages, the assignment operator = behaves differently compared to ordinary operators like + or *, as in assignments like let a = b + c() obviously the right side has to be evaluated before the result can be actually assigned to variable a.

Control Structures

Larger computer programs generally consist not only of code that is executed linearly but contain code for conditional or repeated execution.

The most important control structures of Nim are the if statement for conditional execution, the related case statement, and the while and for loops for repetitions. All these statements control the actual program execution at program runtime. Syntactically very similar to the if statement is Nim’s when statement, which is already evaluated at compile-time, and can be used to adapt our program code for various operating systems or to compile our code with special options, e.g. for debugging or testing purposes.

All these control structures can be nested in arbitrary ways, so we can have in one if branch other if conditions or while loops, and in while loops again other control structures including other loops.

If Statement and If Expression

The if statement with multiple optional elif branches and an optional else branch evaluates a sequence of boolean conditions at program runtime. As soon as one condition evaluates as true the corresponding statement block is executed, and after that, the program execution continues after the whole if construct. That is, at most one branch is executed. If none of the conditions after the if or elif keywords evaluates to true, then the else branch is executed if it exists. A complete if statement consists of one if condition, an arbitrary number of elif conditions, and one optional else part:

if condition1:
  statement1a
  statement1b
  ...
elif condition2:
  statement2a
  statement2b
  ...
elif condition3:
  statement3a
  statement3b
  ...
elif ...:
  ...
else:
  statementa
  statementb
  ...

The most simple form of an if statement is

if condition:
  statement
if age > 17:
  echo "you may drink and smoke, but better avoid it!"

Note that the branches are indented by spaces, we use two spaces generally, but other numbers work as well. And note that it is elif, not elsif like in Ruby and that there is a colon after the condition. Instead of a single statement, we can use multiple in each branch, all on their own line and all indented in the same way.

No, the terminating colon is not really necessary for the compiler, the compiler could determine the end of the condition without it, as the following statement is indented. But it looks better with colon, the colon makes it for humans easier to understand the structure of the complete if statement. So the compiler expects the colons and complains otherwise currently.

When there is no elif and no else part, then we can also write the conditional code directly after the colon, like

if age > 17: echo "you may drink and smoke, but better avoid it!"

With an elif and an else branch, the example from above may look like

var age: int = 7
if age == 1:
  echo "you are really too young to drive"
elif age < 6:
  echo "you may drive a kids car"
elif age > 17 and age < 75:
  echo "you can drive a car"
else:
  echo "drive carefully"

Note that we perform the age tests in ascending order — it would not make much sense to first test for a condition age < 6, and later to test for age < 4, as the if statement is evaluated from top to bottom, and as soon as one condition is evaluated as true, that branch is executed and then the program execution continues after the whole if construct. So a later test age < 4 would be useless when that condition is already covered by a prior test age < 6.

As the various conditions of the if statement are processed from top to bottom until one condition evaluates to true, it can be a good idea to put the most likely conditions first for optional performance, as then the unlikely conditions have not to be evaluated in most cases. Another strategy for larger if/elif constructs is to put the most simple and fast tests to the top when possible.

We can also have if/else expressions that return a value like in

var speed: float = if time > 0: delta / time else: 0.0 # prevent div by zero error

In C for a similar construct, the ternary ? operator is used.

In languages like C or Ruby the assignment operator = is an expression that returns the assigned value, so in C we can write code like

while (char c = getChar()) { process(c)}

In Nim, the assignment operator is not an expression with a result, but we can group multiple statements in round brackets separated by semicolons, and when the last statement in the bracket is an expression, then the whole bracket has the same value. So we can use conditional terms like

while (let c = getChar(); c != '\0'):
  process(c)

If we declare a variable in this way using the var or let keyword, then that variable is only visible in the bracket expression itself and in the following indented block.

Note that if-expressions must always return a well-defined value, so they must always contain an else branch. A plain if, without an else, or an if/elif without an else does not work. And as Nim is a statically typed language and all variables have a strictly well-defined type, the if-expression must return the same type for all branches!

var a: int
var b: bool
a = if b: 1 elif a > 0: 7 else: 0 # OK
a = if b: 1 elif a > 0: 7 # invalid
a = if b: 1 # invalid
a = if b: 1 else: 0.0 # invalid, different types!

The When Statement

The when statement is syntactically very similar to the if statement, but while all the boolean conditions are evaluated during the program run time for the if statement, for the when construct all the when/elif/else conditions have to be constant expressions, and are already evaluated at compile-time. In ordinary program code, the when statement is not used that often, but it is useful when we write bindings to C libraries and low-level code. Common use cases for the when statement are the isMainModule condition test and the test for defined symbols, like defined(windows):

when not defined(gcDestructors):
  echo "You may try to compile your code with option --gc:arc"
when isMainModule:
  doAllTheTests()

The value isMainModule is only true for a source code file when that file is compiled directly as the main module, that is, when it is not indirectly compiled because it is imported by other modules. This way we can include easily test code to our library modules — that test code is ignored when the module is used as a library, but active when we compile the module direct for testing.

A when defined() construct can be used to test for predefined or our own custom options, e.g. we may give the optional option -d:gintroDebug to the compiler and test in the code of that module for this option, like when defined(gintroDebug):.

One difference between the when and the if statement is, that the "then" branches do not open a new scope, so variables which we define there are still visible after the construct has been processed:

when sizeof(int) == 2:
  var intSize = 2
  echo "running on a 16 bit system!"
elif sizeof(int) == 4:
  var intSize = 4
  echo "running on a 32-bit system!"
elif sizeof(int) == 8:
  var intSize = 8
  echo "running on a 64-bit system!"
else:
  echo "cannot happen!"

echo intSize # variable is visible here!

Another peculiarity of the when statement is, that it can be used inside object definitions — we will show an example of that in a later section of the book when we introduce the object data type. In the same way as the if construct, when can also be used as an expression.

The Case Statement

The case statement is not used that often, but it can be useful when we have many similar conditions:

case inputChar
of 'x': deleteWord()
of 'v': pastWord()
of 'q', 'e': quitProgram()
else: echo "unknown keycode"

To enable optimizations, the case construct has some restrictions compared to a more flexible if/elif statement:

The variable after the case keyword must have a so-called ordinal type like int, char, or string, while float would not work. And the values after each of keyword must be constant, that is, a single constant value, multiple constant values, or a constant range like 'a' .. 'd' for the 4 first lower case letters. Of course, these constants must have a type compatible with the type of the variable after the case keyword. A case statement must cover all possible cases, so most of the time an else branch is necessary.

For Nim version 1.6 the case statement can contain also optional elif branches with arbitrary boolean conditions. This was not the case in the wirthian languages Pascal, Modula, and Oberon, and makes Nim’s case construct now very similar to the ordinary if/elif/else.

Unless the similar switch statement in C, the case statement needs no break after each branch. If a condition after an of keyword is true, then the corresponding statement or statement sequence is executed, and after that, program execution continues after the whole case construct.

The case construct can also be used as an expression like in

var j: int
var i: int =
  case j
    of 0 .. 3: 1
    of 4, 5: 2
    of 9: 7
    else: 0

Here, an else is necessary to cover all cases. And as you see, we can also indent the block after the case keyword if we want.

The While Loop

The while loop is used when we want to do conditional repetitions, that is if we want to check a condition and want to execute a block of statements only as long, as the condition is true. If the condition is false in advance or becomes false after some repetitions, then the program execution proceeds after the indented loop body block.

A basic while loop has this shape:

while condition:
  statement1
  statementN
firstStatementAfterTheWhileLoop
var repetitions = 3
while repetitions > 0:
  echo "Nim is easy!"
  repetitions = repetitions - 1

That loop would print the message three times. Like the condition in the if-clause, the condition is terminated with a colon. Note that the condition must change during the execution of the loop, otherwise, when the condition is true for the first iteration, it would remain true and the loop would never terminate. We decrease the loop counter repetitions in the loop, so at some point, the condition will become false, the loop will terminate, and program execution will continue with the first statement after the loop body. Note how we decrement the loop counter: The right site of the assignment operator is evaluated, after that is done, the new value is assigned to the counter.

There exist two rarely used variants of a while loop: the loop body can contain a break or a continue statement, which each consists only of this single keyword. A break in the body stops the execution of the loop immediately and continues execution after the loop body. And a continue statement in the body skips the following statements in the body and starts at the top again, the while condition is evaluated again.

var input = ""
while input != "quit":
  input = readLine(stdin)
  if input == "":
    continue
  if input == "exit":
    break

The above code used the == and the != operators. The == operator does a test for equality, and != test for inequality. Both operators work for most data types like integers, floats, characters, and strings. The literal value of an empty string is written "". In line 2 we test if the variable named input has not the value "quit", and in line 4 we test if that variable is empty, that is it contains no text at all.

The use of break and continue destroys the expected flow in loops, it can make understanding loops harder. So we generally avoid their use, but sometimes break or continue are really helpful. For example, when an unexpected error occurs, maybe by invalid user input.

There is no repeat loop as in Pascal in Nim, which does the first check at the end of the loop when it was executed already for the first time. Repeat loops are not used that much in Pascal, and they are sort of dangerous because they check the condition after the first execution of the body, so potentially the body is executed with invalid data for the first iteration. Later, we will see how we can use Nim macros to extend Nim by a repeat loop that can be used as it would be part of Nim’s core functionality.

The Block Statement

The block statement can be used to create a new indented block, which creates a new scope, in the same way as an if true: statement would do:

block: # create a new scope
  var i = 7
echo i # would not compile, as variable i is undefined

Blocks can be useful to structure large code segments, when there are no better ways, such as splitting the code in multiple procs, available. For testing purposes, blocks can be useful too, to keep the symbols in a local scope. But actually, most useful are blocks, when the blocks get attached names, and we use the break statement in a while or for loop to break out of a nested loop:

let names = ["Nim", "Julia", "?", "Rust"]
block check:
  for n in names:
    for c in n:
      if c notin {'a' .. 'z', 'A' .. 'Z' }:
        echo "invalid character in name"
        break check
echo "we continue"

The break check statement would immediately leave the nested loops and continue with the first statement after the block, which is the last line in the code segment above. Using break in such a way is not very nice, as it may make it harder to understand the code structure, but sometimes it can be very useful.

For Loops and Iterators

For loops can be used to easily iterate over containers, collections, ranges, and many more entities. We have not discussed the important array and seq containers yet, but we know already the string container. The characters of an ASCII string are numbered starting at 0, and we can access them by use of the subscript operator []. So we could print the single characters of a string in this way:

var
  s = "Nim is not always that easy?"
pos = 0
while s[pos] != '?':
  echo "-->", s[pos]
  inc(pos)

It is obvious that the pos variable is some sort of annoying here — we want to process all the characters in the string in sequence, so why would we have to use a position variable to do that. And this way is susceptible to errors, maybe we forget to increase the pos variable in the loop body. So most modern languages provide us with iterators for this purpose:

var
  s = "Nim is not always that easy?"
for ch in items(s):
  echo "-->", ch

That is obviously shorter. The for construct may appear a bit strange, and it is indeed, but it is a common way to write iterations, it is used in Python too. Ruby uses something like s.each{|ch| …​} instead.

For loops can be used to iterate over containers or collections, and pic each element in sequence in this process. The variable after the for keyword is used to access or reference the single elements. That variable has automatically the right type, which is the type of the elements in the container, and gets in each iteration the value of the next element in the container, starting with the first element in the container and stopping when there is no element left. Items() is here the actual iterator, which allows us to access the individual characters in sequence. It exists the convention in Nim, that an items() iterator is automatically called in a for loop construct when no iterator name is explicitly given, so we could also write shorter for ch in s: in this use case.

You may recognize that the output of the above for loop is not identical to the output of the previous while loop. The while loop stops when the last character, that is '?', is reached, while the for loop processes this last character still. That is intended for the for loop, its general purpose is to process all the elements in containers or collections.

The above for loop does a read access to the string, that is, we get basically a copy of each character, and we can not modify the actual string in this way. When we want to modify the string, there is a variant available.

var
  s = "Nim is not always that easy?"
for ch in mitems(s):
  if ch == '?':
    ch = '!'

Here we use mitems() instead of the plain items(), the leading "m" stands for mutable. In the loop body, we can assign different values to the loop variable and in this way modify the container content.

We can not only iterate over containers but over many more entities, for example, over lines of a file or integer ranges. We can use predefined iterators or create our own ones, and then use the iterator in for loops. Iterators are similar to functions, but while functions return only once, iterators can yield results multiple times. Actually, Nim currently provides two types of iterators — inline iterators, which are currently the default type, and closure iterators, which are similar to functions. Inline iterators create a hidden while loop whenever they are called, this way they offer the highest performance, but have some restrictions, and increase the final code size of the executable in the same way as an explicitly while loop would do. Closure iterators are real entities like procs, e.g. we can assign them to variables, but in the for loop, each call generates some minimal overhead. We will learn how to create our own iterators later in the book after we have learned all the details about procedures and functions.

Objects

We have worked with basic data types like numbers, characters, and strings already. Often it makes sense to join some variables of these basic data types to more complex entities. Assume you want to build an online store to sell computers, and you want to build a database for them. The database should contain the most important data of each device type, like the type of CPU, RAM and SSD size, power consumption, manufacturer, quantity available, and actual selling price.

We can create a custom object data type with fields containing the desired data for this purpose:

type
  Computer = object
    manufacturer: string
    cpu: string
    powerConsumption: float
    ram: int # GB
    ssd: int # GB
    quantity: int
    price: float

In the first line, we use the type keyword to tell the compiler that we want to define a new custom type. Writing the type keyword on its own line begins a type section where we can declare one or more custom data types. All type declarations in a type section must be indented. In the next line, we write our type name, an equal sign, and the keyword object. That indicates that we want to declare a new object type named Computer. Here Computer is a type name, in Nim we use the convention that user defined type names start with a capital letter. In the following indented block we specify the desired fields of this object, each line contains the name of a field, and a colon followed by the needed data type. That is similar to a plain variable declaration.

Objects in Nim are similar to structs in C. Unlike classes in Java, Nim objects contain only the fields, sometimes also called member variables, but no procedures, functions, or methods, and no initializers or destructors as in C++. In Nim, we keep the data objects, and the procedures, functions, methods, and also optional initializers and destructors that work with that data objects, separated.

Now, that we have defined our own new object type, we can declare variables of that type and store content in its fields.

var
  computer: Computer

computer.manufacturer = "bananas"
computer.cpu = "x7"
computer.powerConsumption = 17
computer.ram = 32
computer.ssd = 1024
computer.quantity = 3
computer.price = 499.99

Of course, in real applications, we would fill the fields not in this way, but we would maybe read the data from a file, from a terminal, or maybe from a graphical user interface.

It may look a bit ugly that we have to write computer. before each field when we access the fields. Indeed, in recent Nim versions that is not necessary, you may use the with construct now instead.

import std/with
var
  computer: Computer
with computer:
  manufacturer = "bananas"
  cpu = "x7"
  powerConsumption = 17
  ram = 32
  ssd = 1024
  quantity = 3
  price = 499.99

We can use the fields like ordinary variables:

computer.quantity = computer.quantity - 1 # we sold one piece
echo computer.quantity

As you already know, the right side of the assignment operator is evaluated first, then the result is stored in the variable on the left side. But we can also just write computer.quantity -= 1 or dec(computer.quantity).

Objects, like all other data types that we have already used, are value types, which means that when an object is assigned to a new variable, all its components are copied as well. In this way, objects behave like strings — assignment copies the content, and the entities remain independent of each other. We will learn about reference types soon, which behave differently.

To initialize object variables, we can use the object type names as a constructor with a syntax like Foo(field: value, …​). Unspecified fields get the field type’s default values:

var
  computer1: Computer(price: 799.99; quantity: 2)
  comp2: Computer

comp2 = computer1
comp2.price = 999.00

To initialize the variable computer1 we used the constructor syntax. In line five, we use the assignment operator to copy the content of variable computer1 into variable comp2, and finally, we overwrite the price field of comp2. As both variables are distinct instances, the fields of variable computer1 are not modified this way.

Typically, a computer store would offer many different types of computers, so it would make sense to store all the different devices in a container like a sequence, called short seq in Nim. In the next section, we will learn how we can do that.

Arrays and Sequences

Sequences and arrays are homogeneous containers, they can contain multiple other elements of the same data type, while a plain variable like a float or an int only contains a single value. In some way, we could regard objects also as containers, because objects contain multiple fields. The same holds for tuples — tuples are a very simple, restricted form of objects and also contain fields. But more typical container data types are the built-in arrays and sequences, or for example, hash tables, which are provided by the Nim standard library. Arrays, sequences, and hash tables can contain multiple elements, but all elements must have the same data type, which we call the base type.[26] The data type of the base type is not restricted, it can be even again array or sequence types, so we can build multidimensional matrices in this way. Arrays have a fixed, predefined size, they can not grow or shrink during the runtime of our program. Sequences and hash tables can grow and shrink.

Arrays and sequences appear very similar, a sequence appears even more powerful because it can change its size, that is the number of elements that it contains, at runtime, while an array has a fixed size. So why do we have arrays at all? The reason is mostly efficiency and performance. An array is a plain block of memory in the RAM of the computer, which can be accessed very fast and needs not much care by the runtime system. Sequences take much more effort, especially when we add elements and the sequence has to grow. When we create sequences, we can specify how many elements should fit in it at least, and the runtime system reserves a block of RAM of the appropriate size. But when our estimation was too small, and we want to append or insert even more elements, then the runtime system may have to allocate a larger block of memory first, copy the already existing elements to the new location, and then release the old, now unnecessary memory block. And this is a relatively slow operation. The reason why this process can be necessary is, that the initially allocated memory block may not increase in size because the neighborhood in the RAM is already occupied by other data. Now, let us see what we can do with arrays and sequences:

var
  a: array[8, int]
  v = 1
for el in mitems(a):
  el = v
  inc(v)
for el in mitems(a)
  el = el * el
for square in a:
  echo square

In the second line of the code above, we declare a variable named a of array type — we want to use an array with exactly 8 elements, and each element should have the data type int. To declare a variable of array data type we use the array keyword followed in square brackets by the number of the elements, and separated by a comma, the data type of the elements. We can also specify the range of the indices explicitly by specifying a range like array[0 .. 7, int] or array[-4 .. 3, int]. The first specification is identical to the one in the above example program, and the second one would allow us to access array elements with index positions from -4 up to 3.[27] or [int, 8]. It may help to remember that for plain variables the data type comes last also like in var i: int.]

The first for loop of the above program fills our array — that is, for each of the 8 storage places in the array, we fill in some well-defined data. We use the mitems() iterator here because we want to modify the content of our array — we fill in numbers 1 .. 8. In the next for loop, we square each storage location, and finally, we print the content. In the last for loop, we do not modify the content, so a plain items() instead of mitems() would work, but we already learned that we have not to write the plain items() at all in this case.

Sequences work very similar to arrays, but they can grow:

var
  s: seq[int]
  v = 0
while v < 8:
  inc(v)
  add(s, v)
for el in mitems(s)
  el = el * el
for square in s:
  echo square

We start with an empty seq here, and use the add() procedure to append elements. After that, we can iterate over the seq as we did for the array.

In the same way, as we access single characters of a string with the subscript operator [], we can use that operator to access single elements of an array or a seq, like in a[myPos]. The slice operator is available for arrays and sequences too and can be used to extract sub-ranges or to replace multiple elements. As arrays have a fixed length, the slice operator can only replace elements in arrays, but not remove or insert ranges. The first element position is generally 0 for arrays and sequences. Arrays can even be defined in a way that the index position starts with an arbitrary value, but that is not used that often. Whenever you use the subscript or slice operator, you have to ensure that you access only valid positions, that is, positions that really exist. a[8] or s[8] would be invalid in our above example — the array has only places numbered 0 .. 7, and for the seq, we have added 8 values which now occupy positions 0 .. 7 also, position 8 in the seq is still undefined. We would get a runtime error if we would try to access position 8 or above, as well, when we would try to access negative positions. You might think that an assignment for a seq like s[s.length] = 9 is the same as s.add(9), but only the add() operation works in this case.

Note that in some languages like Julia arrays start at position 1.[28] Nim arrays can have an arbitrary integral start position, including negative start positions, but the start position as well as the highest subscript position are determined in the program source code and can not change at runtime. We say that arrays have fixed compile-time bounds. Sequences always start at position 0, we can specify an initial size, and we can always add more elements at runtime.

Arrays and sequences allow fast access to its elements: All the elements are stored in a contiguous memory block in RAM, and the start location of that memory block is well-known. As all the elements have the same byte size, it is an easy operation to find the memory location of each element. The compiler uses the start location of the array or seq, and adds the product of subscript index and element byte size. The result is the memory location of the desired element, which was selected by the index used in the subscript operator. When the array should not start at position 0, then the compiler would have to adjust the index, by subtraction of the well-known start index. This operation takes not much time, but still arrays starting at position 0 may be a bit faster. We said that the compiler has to do a multiplication of index position and element size — that is an integer multiplication, which is very fast. When the element size is a power of two, then the compiler can even optimize the multiplication by using a simple shift operation, which may be even faster, depending on your CPU.

It should be not surprising that the internal structure of sequences is a bit more involved than arrays. Arrays are indeed nothing more than a block of memory, generally allocated on the stack for local data, or allocated in the BSS segment for global data. Don’t worry when you have not yet an idea what the stack, the heap, and a BSS segment are, we will learn that soon. The Nim seq data type has a variable size, so it is clear that it needs not only a storage location for its elements, but also a counter to store how many elements it currently contains, and another counter for how many it could contain at most. The element counter must be updated when we add or delete elements, and when the counter tells that there is currently no more space available for more elements, then a new block of memory must be allocated, and the existing elements must be copied from the old location into the newly allocated memory region before the old memory region can be released.[29] Due to this additional effort appending elements to a seq by using the add() procedure is not extremely fast. You may wonder why we have not to save a size information for arrays. Well arrays have fixed sizes, so it is obvious that we never have to adjust something like a size counter, simple because the size would never change. But would we have to save the desired initial size of the array? Well, in some way yes. But it is a constant value. During the compile process, the compiler can catch some errors already for us — when we have an array as above with size 8, then the compiler would be able already at compile time to recognize some invalid access to array elements — a[9] would be a compile-time error for sure. But at runtime, when we execute our program, access to not existing index position may occur, for example, by constructs like var i = 9; a[i] = 1 when the array is declared as var a: array[8, int]. For catching that type of error, the compiler has to store the fixed array size somewhere and check against that value when an array access by using the subscript operator with a non-constant argument occurs, as the a[i] above. One related remark: Accessing array elements is as fast as ordinary variable access when we use a constant value as index, that is a constant literal or a named constant. The reason for this is, that when the index is a constant, then the compiler just knows the exact position of that array element in memory, just as it knows the address of plain variables, so there is no need for address calculations at runtime. Actually, to access an array element with a given constant index position, the compiler only has to add a constant value to the current stack pointer, as arrays are stored on the stack. To access a constant position in a seq, the compiler would have to add a constant to the base address of the memory block that contains the seq data.

We said that appending elements to sequences is not extremely fast — indeed, it is a few times slower than access to an array element by its index using the subscript operator. So when we know that our seq will have to contain at least an initial amount of elements, then it can be useful for maximum performance, that we allocate the seq from the beginning for this size and then fill in the content by use of the subscript operator instead that we append all the elements one by one. Here is one example:

var s: seq[int] = newSeq[int](8)
var i: int
while i < 8:
  s[i] = i * i
  inc(i)

We use the newSeq() procedure to initialize the sequence for us, the content of the square brackets tells the newSeq() procedure that we want a sequence with base type int, and the number 8 as an argument tells it that the newly created sequence should have 8 elements with the default value (0). This procedure is a so-called generic procedure, it needs additional information, which is the data type that the elements should have. Don’t confuse the square bracket in the newSeq[int]() call with the subscript operator a[i] which we have used for array access, both are completely unrelated. Note that the initialization of the seq above does not restrict its use in any way, we can still use it like an uninitialized seq, that is we can use the add() operator to add more elements, we can insert or delete elements, and all that.

Deleting elements from an array or from a sequence can be very slow.[30] It is slow when we use the naive approach and move all the elements located after the element that should be removed one position forward. This would obtain the order in the container, so sometimes this is the only solution, but of course moving all the entries is expensive for large containers. Nim’s standard library provides the delete() function for this order maintaining delete operation. A much faster way to delete an entry in a seq or array is to remove the last entry and replace the one that should be deleted with that last entry. This operation moves the last entry to the front, so the order of elements is not maintained. Nim’s standard library provides the del() function for this faster, but order changing delete operation. Of course, whenever the order is not important, we should use del(). The delete() and del() functions are actually only available for sequences, as arrays have a fixed size — but in principle, we could do similar operations with arrays as well, we have just to store the actual used size somewhere. [31]

In the section about strings, we said that strings have value semantic, that is, that an assignment like str1 = str2 creates a copy of str2 and that after that assignment str1 and str2 are fully independent entities — modifying one does not change the content of the other one. Array and sequences behave in the same way, both have value semantics too. Indeed, arrays are true value types in Nim, as they live on the stack in the same way as plain variables like integers, floats or characters. Sequences have a dynamic data buffer, which is allocated on the heap, so it would be possible that an assignment like seq1 = seq2 would not copy the data buffer but reuse the old one. In that case, both sequences would be not independent, seq2 would be an alias for seq1. This is called reference semantic, some languages like Ruby behave in this way. But in Nim arrays, strings and sequences have value semantic, an assignment creates an independent copy. We will learn more details about reference semantics and the use of the stack or heap to store data soon when we discuss references to objects.

Some details

Let us investigate at the end of this section some internal details about arrays and sequences. Beginners not yet familiar with the concept of pointers should better skip this subsection and maybe come back later. We could consult the Nim language manual or the compiler source code to learn more details about arrays and sequences. Or we can write some code to test properties and behavior. Let us start investigating an array:

proc main =
  var a: array[4, uint64]
  echo sizeof(a)
  a[0] = 7
  echo a[0]
  echo cast[int](addr a)
  echo cast[int](addr a[0])

  var a2 = a
  a[0] = 3
  echo a2[0]

main()

When we run this program, we get this output:

32
7
140734216410384
140734216410384
7

The size of the whole array is 32, as we have 4 elements, each with 8 bytes in size. And the address of the array itself as well as the address of its first element is identical. Remember that the actual address values will be different for each run of our program, and they may be totally different on different computers, as it is some random choice of the OS which free memory area is used to run our program. This result is expected as the array is a plain block of memory stored on the stack. And indeed the array has copy semantic, when we create a copy called a2 and later modify a, then the content of a2 is unchanged. That was not really surprising, so let us investigate a sequence:

proc main =
  var dummy: int
  var s: seq[int64]
  echo sizeof(seq)
  echo sizeof(s)
  s.add(7)
  echo s[0]
  echo cast[int](addr dummy)
  echo cast[int](addr s)
  echo cast[int](addr s[0])

  var s2 = s
  s[0] = 3
  echo s2[0]

main()

When we run the above code, we get:

8
8
7
140732171249104
140732171249112
140463681433696
7

The first two lines of the output may confuse us, as a size of only 8 bytes may indicate a plain pointer value on a 64-bit system. Indeed, the sequence is not a large object that contains size and capacity fields, but only a tiny object that contains a single pointer to the data storage of that sequence. We know that it is not a plain pointer or ref by the fact that we can not assign nil or test for nil for sequences. (But an object which contains only a pointer is basically identically to a plain pointer, as Nim objects have no overhead as long as we do not use inheritance and when no padding to word size is needed for tiny fields like int8.) Capacity and length are stored also in the memory block that is allocated for the elements, as long as the sequence is not empty. So empty sequences don’t waste much memory when we have a lot of them, i.e. arrays or sequences of sequences (matrices). We use the dummy int variable in the code above as we know that plain ints are stored on the stack, and when we compare the addresses of our dummy variable and our sequence, then we see that the addresses indicate close neighborhoods, so the seq object is also stored on the stack. But the address of s[0] is very different, indicating that the data buffer is stored in a different memory region, which is the heap. If we would continuously add elements to the seq, then the address s[0] would change at some point, while the address of s would always remain unchanged. That is because the capacity of the data buffer would become exhausted at some point and a new data buffer with a different address would be used. Finally, we see again that the sequence has also copy semantic, as the content of the copy s2 remains unchanged when we modify the initial sequence s. We could try to discover some more details of the internals of Nim’s sequences, i.e. we could try to detect where the capacity and size are stored. But that are internal details that should not really interest us and which may change with new compiler versions or different compilers.

But OK, you may still not believe what we said, so let us go one layer deeper. We strongly assume that a seq needs a length and a capacity field. And we assume that its data type should be int. We said that both fields should be adjacent to the buffer of the seq elements, which means at the start or at the end. Obviously, we can not access the end as long as we do not know the capacity, so the capacity field should be at the start, and then the length field also. We may find out which one is which by observing the content when seq grows. So let us write some code:

proc main =
  var
    s: seq[int64] = newSeqOfCap[int64](4)
    s2: seq[int64]
    p: ptr int

  var h = cast[ptr int](addr s2) # prove that an uninitialized seq is indeed a pointer with nil (0) value
  echo cast[int](h) # address on stack
  echo h[] # value (0)
  echo ""

  for i in 0 .. 8:
    s.add(i)
    echo cast[int](addr s[0])
    p = cast[ptr int](cast[int](addr s[0]) - 8) # capacity
    echo p[]
    p = cast[ptr int](cast[int](addr s[0]) - 16) # length
    echo p[]

main()

The output when we run the program is:

140725732630192
0

140251431497824
4
1
140251431497824
4
2
140251431497824
4
3
140251431497824
4
4
140251431506016
8
5
140251431506016
8
6
140251431506016
8
7
140251431506016
8
8
140251431510112
16
9

Don’t worry when you do not understand the program and the output yet. You will better understand it when you have read the sections about references, pointers, and memory management. The first two output lines show us that an uninitialized seq is just a pointer pointing to nil. And the remaining output lines show us the address of the first seq element, the capacity, and the length of the seq whenever we add an element. We started with a seq with an initial capacity of 4, so address and capacity are constant while we add the first 4 elements. Then the capacity of the allocated buffer is exhausted. A new buffer with a different address and doubled capacity is allocated, the already contained elements are silently copied to the start of the new buffer, and so on.

Slices

Nim slices are objects of type Slice with two fields, a lower bound a and an upper bound b. The system module also defines the HSlice object, called heterogeneous slice, for which the lower and upper bound can have different data types:

type
  HSlice*[T, U] = object   ## "Heterogeneous" slice type.
    a*: T                  ## The lower bound (inclusive).
    b*: U                  ## The upper bound (inclusive).
  Slice*[T] = HSlice[T, T] ## An alias for `HSlice[T, T]`.

As the Slice and HSlice objects are not built-in types, their names start with capital letters. Slices are not used that often directly, but mostly indirectly with the .. range operator, e.g. to access sub-ranges of strings and other containers.

One example of its direct use from the system module is

proc contains*[U, V, W](s: HSlice[U, V], value: W): bool {.noSideEffect, inline.} =
  result = s.a <= value and value <= s.b

Slices are used by functions of the standard library or by user-defined functions to access sub-ranges of strings, arrays, and sequences. Typically, we do not use an explicit Slice object, but we create the Slice by use of the infix .. operator, which takes two integers and returns a Slice with these bounds:

Applied to container data types, slices look syntactically like sub-ranges:

var m = "Nim programming is difficult."
m[19 .. 28] = "not easy."
echo m
echo "Indeed " & m[0 .. 18] & "is much fun!"
var s = HSlice[int, int](a: 0, b: 18)
echo "Indeed " & m[s] & "is much fun!" # the same as line four

In line two we use the slice to replace the sub-string "is difficult.", which starts at position 19, with another string. Note that the replacement can be a longer or a shorter string, that is, the slice supports not only overwriting characters but also inserting or deleting operations. In line two, the actual Slice object is constructed by the .. operator and the two integer bounds. In line four, we use the slice to access a sub-string and create a new string with it. As we learned earlier in the Strings section already, we can use the ^ operator to access elements counted from the end of the container, so we could have written line two also as m[19 .. ^1] = "not easy.". The last two lines in the above example show that we could have used a real HSlice object to access the sub-string instead.

Slices can be used in a similar way for arrays, strings, and sequences. But we have to remember that Slices are only objects with a lower and an upper bound, so there must be always a procedure that accepts the container and the Slice as arguments to do the real work.

When we care for utmost performance, then we have to be a bit careful with Slices, as the use of Slices can generate copies. Consider this example:

type
  O = object
    i: int

proc main =
  var
    s = newSeq[O](1000000)
  for i in 0 .. (1000000 - 1):
    s[i] = O(i: i)

  var sum = 0
  for x in s[1 .. ^1]:
    sum += x.i

Here, we use the slice construction operator .. to exclude the first element from our summing operation. Unfortunately, when we use the slice operation in this way, the Nim compiler may create a copy of our sequence, which increases the run-time and memory consumption. At least for Nim versions up to 1.6, this was the case, never versions may use view types instead to avoid the copy. We may try to use the new toOpenArray() expression and try a construct like

  for x in items(s.toOpenArray(1, s.high)):

but that does currently not compile.

One option is currently that we create a custom iterator like

iterator span*[T](a: openArray[T]; j, k: Natural): T {.inline.} =
  assert k < a.len
  var i: int = j
  while i <= k:
    yield a[i]
    inc(i)

and use

for x in s.span(1, s.high):

Or we may do the summing in a procedure and pass that proc an openArray created with toOpenArray() like

proc sum(x: openArray[O]): int =
  for el in x:
    inc(result, el.i)
echo sum(s.toOpenArray(1, s.high))

But this is a work in progress, so the situation may improve, see

Value Objects and References

We have already used different types of variables — integers, floats, characters, the custom Computer object, and some more. We said that variables are named memory regions, where the content of our variables is stored. We call these types of variables also value types.

Value types always imply copies when we do an assignment:

var i, j: int
i = 7
j = i
i = 3
echo i, j

Here we have 3 assignments, first, we assign the integer literal 7 to the variable i, then we assign the content of variable i to variable j, and finally, we overwrite the old content of variable i with the new literal value 3. The output of the echo() statement should be 3 and 7, because in line 3 we copy the content of variable i, which is currently the value 7, into variable j. The new assignment in line 4 in no way touches the content of variable j.

In section Objects we saw that the fields of object types like our Computer data type behave in the same way — assignments copy the content. The tuple data type, which has some similarities to objects, and which we will introduce later in the book, behaves the same. All these data types are stack-allocated, and we say that the data types have value or copy semantic. Even strings and sequences, which use actually a heap-allocated data buffer, behave in the same way in Nim.

Whenever possible, we should use this simple form of variables, as they are fast and easy to use.

Maybe that is not too surprising for you, but when we would have references instead of plain variables, then the situation would be different, as we will see soon. Actually, some other programming languages use reference semantics for entities like strings by default, for example in Ruby an assignment of a string variable to another variable does not copy the content, so that both variables still use the same data buffer — when we then modify one variable, the content of the other changes too.

But there exist situations where we need some sort of indirection, and then references and pointers come into play. For example, when the data entities depend in some form on each other, the elements may build linked lists, trees, or other structures. The entities may have some neighborhood relation, also called some one-to-many relation.

Indeed, value objects and references occur in real life also:

Imagine you have baked a cake for your family, and you know that your friendly neighbor loves cakes too. As you have still a lot of all the necessary ingredients and because the oven is still hot, you make one more identical cake to give it later to your neighbor. We can think of the cake as a value type, and your second cake can be considered as a copy. When you give the copy to your neighbor, then you have still your own, and when you or the neighbor eats the cake, then the other one still exists.

Now imagine that you know a good car repair shop. You can give the telephone number or location of that car repair shop to your neighbor, so he can use that shop too. So you gave him a reference to the shop, but you gave him not a copy. You can also give some of your other friends each a reference to that shop, which is nearly no effort for you. While backing a cake for all of them would be some effort. But there is some danger with references: When one of your friends gets angry and burns down the car repair shop, then you and all your other friends have a serious problem.

You can regard the names of persons as some sort of reference too. Imagine you have a list with the names of all the people you intend to invite to your birthday party and another list with the names of people who owe you money. Some names may be on both lists, this is it refers to the same person.

In computers, the dynamic storage, called RAM, consists of consecutive, numbered storage locations, called words. Each individual word has its address, which is a number typically starting at zero and extending to a value, which is defined by the amount of memory available in your computer. These addresses can be used to access the storage locations, that is, to store a value at that address, or to read the content again. Reading generally does not modify the content, you can read it many times and will always get the same value. When you write another value to that storage location, then reads will give you that new value.

Basically, for all the data that you use in your program you need in some form its address in the RAM, without the address, you can not access it. But what is with all the plain, value object variables we have used before, we have never used addresses? That is true — we used only names to access our variables, and the compiler mapped our chosen name to the actual address of the variables in memory whenever we accessed the variable. For most simple cases, this is the best way to access variables. Now, let us assume we have such value object type of variable declared in our program, can we access it without using its name? When we have declared it, it should reside somewhere in the RAM when the program is executed. Well, when we do really not want to access it by variable name, then there is still one chance: We can search in the whole RAM for the desired content. In practice, we would never do that, as it is stupid and would take very long, but we could do it. But how can we detect our variable? How can we be sure that it is indeed ours? Generally, we can not. Even when we are sure that the variable must reside somewhere in the RAM, typically the variable is marked in no way, of course. Even when we would know the value, which is stored in that variable, we would only know what bit pattern it should have, so for most words of the RAM with a different bit pattern we could say for sure that it can not be our variable, but whenever we find the expected bit pattern then it can be just a coincidence, there can be many more words in RAM with that content. In some way, it is as if you would search for a person, and you know that that person lives on a long road with numbered houses. If you only know that the person wears brown shoes, but you know not the number of the house nor the name of the person and no other unique property of that person, then you have not much luck.

References and Pointers

Introduction to Pointers

In Nim references are some form of smart or managed pointers, we will learn more about references later. The plain pointer data type is nothing more than a memory address, it is similar to a (unsigned) integer number. We say that a pointer points to an entity when the pointer contains the memory address of that entity.

Besides the pointer data type, which is only some RAM address, we have also the ptr entity. Ptr is not a datatype on its own, it is always used in conjunction with another data type:

var
 p: pointer
 ip: ptr int

Here the variable p is of type pointer, we could use it to point to some arbitrary memory address. The variable ip is of type ptr int, which indicates that it should only point to memory addresses where a variable with data type int resides. So a ptr is a pointer that is bound to a specific data type. Generally, we speak only about pointers, if we are referring to an untyped pointer or a typed ptr is generally clear from the context.

When we only declare pointers but do not assign a value, then the pointers have the value nil, which indicates that they are regarded to point to nothing. Exactly speaking, a pointer can never point to anything in the same way as an integer variable can not contain any number. As an integer variable always contains a bit pattern, a pointer also always contains a bit pattern. But we are free to define a special pattern as nil, and whenever a pointer has this special value, then we know that it does not really point to something useful. In C instead of nil, NULL was chosen for the same purpose. In practice nil and NULL are typically mapped to 0, that is, a word with all bits cleared. But that is more or less an arbitrary decision.

So how can we give our pointers above a useful value?

One possibility would be to use Nim’s addr() function, which gives us the memory address of each ordinary variable.

var
 number: int = 7
 p: pointer
 ip: ptr int
echo cast[int](p)
echo cast[int](ip)
p = addr(number)
ip = addr(number)
echo cast[int](p)
echo cast[int](ip)

First, we declare an ordinary integer variable called number which will reside somewhere in memory when we execute the program, and then we use the addr() function to assign the address of that variable to p and ip. The addr() function is a low-level function provided by the compiler, it can be used to determine the memory address of variables and some other entities known to the compiler.[32] We used the echo() procedure to show us the numeric decimal value of the addresses in the terminal. As it typically makes not too much sense to print addresses, echo() would refuse to print it, so we have used the construct cast[int](someValue) to tell that echo() should regard our pointers as plain integer and print it. That operation is called casting, we mostly should avoid it, as it destroys type safety, but for learning purposes, it is OK to use it. We will learn more about casts and related type conversion later.

The first two echo statements should print the decimal value 0, as the pointers have the initial default value nil.

The echo()s in the last two lines should print a value different from 0, as we have assigned the valid address of an ordinary variable that resides somewhere in the RAM when the program is executed. Both outputs should be identical, as we have assigned for both pointers addr(number) each.

Maybe a funny fact is, that when you run the program multiple times, the output of the last two echo() statements print different values. But that is not really surprising — whenever you launch the program, then for our variable number, a storage location in RAM is reserved. And that location can differ for each new program execution. For your next holiday in the same hotel, you may get a different room also.

So when we have the pointer ip pointing to a valid address, can we recover the content of that memory region? Sure, we use the dereference operator [] for that purpose. Whenever we have a typed pointer x we can use x[] to get the content of the memory location where the pointer is pointing to. Note that the operator [] is not really related to the subscript operator [pos] which we used earlier for array, seq, and string access. Nim uses ASCII characters for its operators, and that set is not very large. And maybe it would even be confusing when we would have a different symbol for each operator. We can consider [] as some form of content access operator — mystring[pos] gives us the character at that position, and ip[] gives us the content of the memory location where ip points to.

var
 number: int = 7
 ip: ptr int
echo cast[int](ip)
ip = addr(number)
echo cast[int](ip)
echo ip[]

What do you expect as output for the last echo() statement? Note that for the last echo() statement we do not need a cast, as ip[] has a well-defined type: ip has type ptr int, so ip[] is of well-defined type int, and echo() can print the content.

Now, let us investigate how we can use pointers to modify the content of variables:

var
 number: int = 7
 ip: ptr int
ip = addr(number)
echo ip[]
ip[] = 3
echo ip[]
echo number

What do you expect for the output of the last echo() statement? Well, remember, ip points to the location where the variable number is stored in RAM. So echo ip[] gave us the content of the number. Now ip[] = 3 is an assignment, the right side of the assignment operator is the literal number 3, which is a value type. Earlier we said that for value types an assignment is a copy operation, the right side of the assignment operator is copied into the variable on the left side. Now ip[] stands exactly for the same content as the name number, and so assigning to ip[] is the same as assigning to number.

Pointer Arithmetic

In low-level programming languages, pointer arithmetic can be useful. For example, old C code often iterates with pointer arithmetic over arrays by use of constructs like sum += *(myIntPtr++). This was done to maximize performance. Modern C compiler generally understands statements like sum += el[i]; i++ well and generates very good assembly instructions for it. So pointer arithmetic is not necessary in C that often today.

Nim does not provide math operations for pointers directly, but we can always cast pointers to integers and do arbitrary math. And of course, we could define our own operators for that purpose, but typically we should avoid that, as it is dangerous, error-prone, and generally not necessary. As an example, let us sum up some array elements:

proc main =
  var
    a: array[8, int] = [0, 1, 2, 3, 4, 5, 6, 7]
    sum = 0
  var p: ptr int = addr(a[0])
  for i in a.low .. a.high:
    echo p[]
    sum += p[]
    echo cast[int](p)
    var h = cast[int](p); h += sizeof(a[0]); p = cast[ptr int](h)
    #cast[var int](p) += sizeof(a[0]) # this compiles but does not work currently

  echo sum
  echo typeof(sizeof(a[0]))

main()

When we do pointer arithmetic or similar math to calculate the address of variables in the computer memory, then memory addresses are used like integer numbers, and so it makes some sense that Nim’s integers have the same byte size as pointers.

References:

Allocating Objects

In the previous section, we learned the basics of pointers. We used the addr() operator to initialize the pointer by assigning the address of an already existing entity. This is in practice not that often done, and it can be a bit dangerous, as it is not always guaranteed that the variable on which we applied addr() will exist as long as our pointer exists. So the pointer may point later to a memory location that is already freed or used by a totally different object. So the use of addr() is more reserved for advanced programmers, who know well what they do, and most of the time addr() is not necessary at all or is only necessary for really low-level code, maybe when interfacing with external libraries written in C. Instead of using addr() to assigning to pointers a valid address, often procedures like alloc() or create() are used to reserve a block of memory:

var ip: ptr int
ip = create(int)
ip[] = 13
echo ip[] * 3
var ip2: ptr int
ip2 = ip
echo ip2[] * 3
dealloc(ip)

Here the procedure create() is used to reserve a block of memory, the int parameter ensures that the block has the size of an integer value. After ip has a valid value, we can store a value in that memory location and read it again. Note that multiple pointers can point to the same memory location: We declared one more int ptr called ip2. But for that pointer, we do not allocate a new block, but we assign the old block that we allocated for ip to ip2. Now both pointers point to the same object, the int value 13. We may call ip2 an alias, as it is a different way to access the same entity.

When we use alloc() or create() to allocate memory blocks, then we have to deallocate them when we need them not anymore. Otherwise, that memory blocks couldn’t be reused. If we would continuously allocate memory blocks and never deallocate, that is, free them, then at some point in time all memory would be occupied — not only for our own program but for all programs running currently on the same computer. We had to terminate our program — when a program is terminated then all resources get freed automatically by the OS.

The use of procedure pairs like alloc() and dealloc() is common practice in low-level programming languages like C, but it is inconvenient and dangerous: We can forget to call dealloc() and waste resources, or we may even deallocate memory blocks, but still use it by our pointers. The latter would at some point in time crash our program, as we would use memory blocks that are already released and may be used for other variables — from our own program or from other programs. Note that in the source code above, there is only one single dealloc() call. The reason for that is, that we only allocated one single memory block in one single create() call, ip2 is only one more pointer that points to that block. If we had used an additional dealloc(ip2) call, then that would be a so-called double-free error.

As you see, using pointers is inconvenient and dangerous. But still, there are situations where plain value type variables do not suffice. The solution of many higher-level programming languages to this problem is a Garbage-Collector (GC). The GC does the dangerous and inconvenient task of deallocating unused memory blocks for us automatically.

To distinguish the GC managed "pointers" cleanly from the manually managed ones, we call them in Nim references, in some other languages they are called traced pointers. References are always typed like ptr, there is no equivalent to the untyped pointer type for references.

For References, we have still to do the allocation ourselves, then we can use the references, and when we are not using them anymore, then the GC frees the corresponding memory block. A typical scenario is that we use references in a procedure or in an otherwise limited block of code: We declare the reference in that code block, allocated and use it, and when the code block is left, the GC frees the allocated memory for us. You may think that the fact that we still have to allocate the memory for our references ourselves is still a concern, as we may forget that step. Well, it is not that dangerous, when we forget the allocation step, we would use a reference with the value nil, which would immediately result in a runtime error. So we would see the problem immediately. Other pointer errors, like missing de-allocation or use after free, are not that obvious and more dangerous. In languages like C tools like Valgrind are used to check for errors like "use after free". Valgrind is a very helpful tool, but it can not find all errors that may occur, and its reports can be very verbose. We may use Valgrind as well when we compile our Nim program with --gc:arc and -d:useMalloc — this can be used to ensure that our program really works perfectly, maybe when we have to use C libraries, and it may help us to find the cause for bugs.

With references, we can rewrite our previous example code in this way:

var ip: ref int
new(ip)
echo ip[] # zero
ip[] = 13
echo ip[] * 3
var ip2: ref int
ip2 = ip
echo ip2[] * 3

We have replaced ptr by ref, and instead of alloc() or create() we are using the new() proc which gets the uninitialized ref as a parameter and allocates a managed memory block for it, that is after the new() call ip has a well-defined value referring to a managed memory block that can store an integer value. The content of that memory block is cleared initially, so echo ip[] would give zero. Again, we can use one more reference ip2, and assign that ref the value of the other, so now both references the same memory block. The advantage here is that we don’t have to care about freeing that block, the GC will do that when appropriate.

To verify that in the example code above, both references really reference the same object in memory, we could add two more lines of code:

ip2[] = 7
echo ip[]
echo ip2[]

Here, we are using the reference ip2 to assign to the memory block the literal value 7. After that assignment, both echo() statements would display that new content.

Using references and pointers to store basic data types like integers is not done that often, in most cases we work with larger objects, and we create some relations between the objects. We will try that in the next section.

References to Objects

You should still wonder for what references are really useful — they seem to be only a more complicated version of plain value type variables.

Now, let us assume we want to create a list of things or persons, maybe a list of our previously used Computer data type, or perhaps a list of persons we will invite to our next party. We will create the party list for now, as the Computer data type we used before has already many fields, and filling all the fields would be some effort, so let us use a new Friend data type which should store only the friend’s name for the beginning — we may add more fields later when necessary. So we may have

type
  Friend = object
    name: string

With that declaration, we could declare a few friends variables like

  var harry, clint, eastwood: Friend

But that is not what we want, we would need a list with all of our friends that we would like to invite to our party, we would want to add friends to the list, and potentially we would want to delete friends also. You may think we could use Nim’s sequence data type for that, and you are right. But let us assume we could not use that predefined Nim data type for some reason. Then we could create a list of linked references to Person.

type
  Friend = ref object
    name: string
    next: Friend

Now our Friend data type is a reference to an object, and the object itself has an additional next field, which is again of type Friend.

That is some sort of recursion. If that should appear too strange, then imagine you have some numbered paper cards, each with two fields: One field name, one field next: In the name field you can fill in a name of a friend, in the next field you fill in the number of the next card. The last card in the chain gets no entry in the next field.

In languages like Nim or C lists, also called linked lists, are dynamically created data structures consisting of elements (called nodes), where each node has a field, which is a reference or pointer to its successor or predecessor. When the nodes have only a successor field, then we call the list a single linked list, and when it also has a predecessor field, then we call it a double linked list. Contrary to arrays and Nim’s sequences, lists do not allow access to arbitrary elements, we can only traverse the list starting from its first element for single linked lists, or also from its last elements, for double linked lists. The first element of a list is also called its head, and the last element is called it’s tail. Often, the head and the tail elements are just plain nodes, but the head can be also an extended node object with additional fields, carrying information for the whole list, maybe an additional string field for the list name. In this section, we use the simplest form of a list, which is a single-linked list, where the head is just an ordinary node. If the head has the value nil, then the list is empty.

Now we create a small Nim program, which reads in names of our friends from the terminal, creates a list of all friends, and finally prints the list.

type
  Friend = ref object
    name: string
    next: Friend

var
  f: Friend # the head of our list
  n: string # name or "quit" to terminate the input process

while true:
  write(stdout, "Name of friend: ")
  n = readline(stdin)
  if n == "" or n == "quit":
    break
  var node: Friend (1)
  new(node)
  node.name = n
  node.next = f
  f = node

while f != nil:
  echo f.name
  f = f.next
1 The actual name for this temporary variable is arbitrary, we could have used el for element, maybe.

This example code seems to be not that easy. But it is not really difficult, and when you have understood it, you can call yourself a Nim programmer already. Perhaps you should think about the code above for a few minutes, before reading the explanations below.

First, let us summarize what our program should do: It should read in some names of our friends, which we would like to invite to our next party. Of course, when entering the names, we would need a way to tell that we are done. In our program we can do that in two ways, we can enter an empty name by just pressing the return key, or we can enter the text "quit" to stop the loop. Unfortunately, that means, that we can never invite a friend with that name to our parties. When we have terminated the input loop, then the next loop prints all the entries to the terminal.

Let us start with the type and variable declarations: We use a user-defined type named Friend, which is a reference to an object, that object type has a field name of type string, and a field next, which is again a reference to the same data type.

We are using two variables, one called n of type string to read in a name or the quit command from terminal, and a variable called f of type Friend. The variable f seems to match only to one single friend, but as the type of f has a next field, it can be a whole list of friends, with f being the start or head of that list.

In the code above, we are using a special while loop — special because the construct while true: and because the loop contains a break statement. Earlier, we said that we should avoid the break statement in loops because it interrupts the control flow and can make it more difficult to understand and prove the flow. But in this case that form makes some sense: For the first loop, we have to first read in a name from the terminal, and then we can decide what to do, so we can not really evaluate a condition after the while statement at the top. So we use the simple constant condition true, which would never terminate the loop. We need a break inside the loop body to terminate the loop.

Let us investigate the second loop first, as it is really easy: In the while condition, we check if the current value of f is nil, that is if there are no more entries in our list. For that case, we terminate the loop, as we are done. If f has not the value nil, then f points to a valid content, that is, there is at least a valid name, which we access by the field access operator and print it with echo f.name. Note that in Nim the field access operator . works in the same way for value object types as well as for ref object types. For ref objects types we could also write f[].name instead of plain f.name, that is we first apply [] to f to get the content, and then use the . operator to access the name field. In some other languages like C, we would have to use a special operator -> to access fields of pointer or reference types.

The most interesting statement in the output loop is f = f.next. We assign the content of f.next to f and proceed with that new content. The content could be a valid reference to one more Friend object, or it could be nil, indicating that our loop should terminate.

The input loop is also not that complicated: To make the process of adding more friends to the list easy, we always add the new names at the beginning. First, we ask the user to enter a name. We use write(stdout) for this, as echo() always generates a new line, but we want to read in the name on the same line. If the name is empty or has the special value "quit", then we terminate the input loop. In the loop we are using a temporary variable called node of type Friend, we allocate a memory block for it with new().[33] Then we assign the read in friend’s name n to the name field. The last two statements of the loop body are a bit demanding: First, we assign to node.next the value of f. Now, node is basically the start of our list, and its next field refers to the first element of the current list. Fine, but we said that the node variable is only a temporary variable, we do not intend to use it longer as necessary. But currently, node is so useful, it is the head of our list. On the other hand, the former list start f is now useless, current f is identical with node.next. So the trick is, we just assign to f the value of node. Now f is the complete list, and we do not need node anymore. The node variable can be used in the next loop iteration again, but we have to allocate a new memory block for the node reference, as the previous memory block is still in use, it contains the name which we just entered and also a reference to the next object in the list.

Note that we add the new elements at the top of the list in this way. We have done it that way because it is very easy in this way. For adding at the end of the list, we would have to use one more reference variable which allows us always access to the current end of the list, or we would have to traverse the list from head to tail whenever we would like to add elements at the tail.

For one more exercise, let us consider deleting entries in our list. Basically, that operation is very easy, we would just skip one entry. Let’s add this code to the program above:

while f != nil:
  write(stdin, "Name to delete: ")
  n = readline(stdin)
  if n == "" or n == "quit":
    break
  if f.name == n:
    f = f.next
  else:
    while f.next != nil:
      if f.next.name == n:
        f.next = f.next.next
        break
      f = f.next

Here we are using again an outer while loop to read in the names which we want to delete. That loop uses the condition while f != nil: because when the list is empty we should stop, of course.

In the loop body we have an if statement, and in the else branch of the if statement we have one more loop. The reason why we need the if statement is, that the case that our name to delete is the first in the list is some sort of special. Let us investigate the inner loop first. That loop assumes that there are at least 2 elements in the list, f, and f.next. We compare the name of the next entry with n. If they match, then we would have to skip the next entry. We can do that by the statement f.next = f.next.next. That is, we replace the reference from the current element f to the next list entry, that is f.next, by the next entry of the next element, which is (n.next).next. We do not have to write the parenthesis. The n.next.next entry can be nil, in that case, it is the end of the list. If we found a matching name, then we terminate the inner loop with a break statement, and we are done. Otherwise, we assign to f the value of f.next and continue the loop execution. Now to the special case where the name to delete is the first in the list. We need the first if branch for that — if already the first element matches the name to delete, then we just skip the first element by setting the head of the list to the next entry, which may or may not be nil.

This is one way to solve the task, for operations on lists there exist in most cases various solutions, some optimized for easy or short code, some for performance. You may copy the code segment above to the end of the former code, and maybe add one more copy of our printing loop at the end again. Then you should have a program that reads in a list, prints the contents, then asks for names to delete, and finally prints the resulting list. Perhaps you can improve the code, or maybe you can detect special corner cases where it may fail. What is, for example, when some of your friends have the same name? May the program fail in that case? Or you may add more fields to your Friend data type. Possibly a textual field with content male or female, and you can report the ratio of male to female. And potentially remove males from the list when we have more males than females?

For references to objects, the assignment operator = copies the references, but not the object. In the same way, the operator == for equality tests compares the references, but not the content of the objects to which the references point. If you want to compare the content of the objects, you can apply the dereference operator [] on both references:

type
  RO = ref object
    i: int

var
  ro1 = RO(i: 1)
  ro2 = RO(i: 1)
  ro3 = ro1

echo ro1 == ro2 # false
echo ro1[] == ro2[] # true
echo ro1 == ro3 # true

Procedures and Functions

Procedures and functions, called proc and func in Nim, are used to structure the program source code. Functions are a subtype of procedures, that do return a value but do not modify global variables or otherwise change the state of the program. When we talk about procedures in this book, then what we say applies to functions as well, unless stated otherwise. Usually, procedures and functions are used to group sequences of statements that perform a specific task. We can pass parameters to procedures, e.g. data that the procedure should process, and the procedure can return a result. Related sets of procedures can be grouped into library modules, e.g. procedures that perform various string operations. We will discuss the use and creation of modules later in the book.

The terms procedure and function were used in Pascal and the other languages of Wirth already, while C uses the term function only, and Fortran uses the term subroutine instead. And finally, Python and Ruby are using the really strange terms def and fun for it.

Nim’s procedures are basically similar, but much more advanced than its equally named cousins in the wirthian languages, or the plain functions in the C language: Nim’s procedures support generics and overloading, named parameters and default values, the special parameter types varargs and openArray, various ways to return a result, and finally multiple calling conventions including the method and command calling conventions.

Introduction

We call or invoke a proc by just writing its name followed by a parameter list enclosed in parentheses. The parameter list can be empty. When we call a proc, then the program execution continues with that procedure, and when the execution of the procedure terminates, then the next statement after that proc call is executed. Sometimes we say that we jump into a procedure and jump back when that procedure terminates.

In Nim, functions are a special form of procedures that return a result and do not modify the current state of the program. Modifying a global variable or performing an input/output operation would be examples of modifying the state. We have already used some predefined procedures like echo() for output operations, add() for appending single characters to strings, and readLine() for reading in textual user input. And we talked about math functions like sin(), cos(), pow() — these are functions as they accept one or two arguments and return a result, but do not change a state — calling them again with the same arguments would always give the same result. ReadLine() is only a procedure, not a function, as the result may be different for each call, and as we pass a file variable as an argument, which may change its state for each call, maybe because the file end is reached. A function is only a special subtype of a procedure, the func keyword indicates to the reader of the code and to the compiler some special properties, that is, that a result is returned and that the global state is not changed. Whenever the func keyword is used, a proc would do as well, and in this text, we mostly speak about procedures, even when a function would do.

Let us start with a very simple function called sqr() for square.

func sqr(i: int): int =
  i * i

A procedure declaration consists of the keyword proc, a user-selected name, an optional parameter list enclosed in parentheses, and an optional colon followed by the result data type. For a function declaration, we use the keyword func instead of proc, and as functions do always return a result, we have always to specify the result data type.

Note that this is only a declaration so far — the compiler could recognize the construct, its parameters, and its result type. Sometimes we call this construct a procedure header.

Typically, we do not only declare a function, but we define it, that is, we add an equal sign to the procedure header and add an indented procedure body that contains the code that is performed for each invocation.

Pure proc declarations can be necessary for rare situations, maybe when two procedures call each other. In this case, the procedure defined first would call the other procedure, which is not already defined, so the compiler may complain about an unknown procedure. We could solve that problem by first declaring the second procedure so that the compiler would know about its existence. We would then define that second procedure later, closer to the end of the program file.

The sqr() proc above accepts an integer argument and returns its square of the same data type. We would call that proc like

var j: int
j = 7
echo sqr(j)

Earlier in this book, we said that the compiler processes our source code from top to bottom and that the final program is executed from top to bottom too. The first statement is indeed true, for that reason, it can be necessary to declare a function at the top, and define it below, as we can not call a proc before it is declared or defined.

For the program execution, we have to know that procs are only executed when we call them. That is, when we write a proc at the top of our source code, then that proc is processed by the compiler, but it is not executed during program runtime before we call it. Actually, as the Nim compiler supports "death code removal", code of procedures that we never call would not make it into our final executable at all.

The procedure body builds a new scope. We can declare entities like variables, constants, types, or other procedures and functions in that scope. These entities are only visible in the procedure body, but not outside of the proc. Will will learn more about scopes and visibility soon.

Parameter lists of procedures consist of one or more lists of parameter names, separated with commas, followed by a colon and the data type of the parameters. The sub-lists with the same data type are separated by semicolons:

proc p(i, j, k: int; x, y: float; s: string)

While the wirthian languages would require semicolons to separate the parameter blocks, in Nim we could also use plain commas for that. For the data types of proc parameters and as the result type all of Nim’s data types are allowed, including structured types, ref, pointer, and container types. Additionally, we can use the data types openArray and varargs as parameter types — these two types are not allowed for ordinary variables, and varargs is not valid as a result type. Will will learn the details of all these types soon. When we call or invoke a procedure, we can pass literal values, named constants, variables, or expressions to it.

When we call a proc with multiple arguments, we have to specify the arguments in the order as they are listed in the proc header, separated with commas, and the arguments must have compatible data types:

var i: int = 7; x: float = 3.1415
p(i, 13, 19, x, 2.0, "We call proc p() with a lot of parameters")

Here compatible data types mean, that for the i, j, and k parameters, which are specified as int types in the proc definition, variables of smaller int types like int16 would work. For the two parameters of float type, we would have to pass floating-point variables or a float literal. As a special case, an int literal would work also, as the compiler knows the desired data type and automatically converts the int literal into a float for us, as long as that is possible without loss of precision. We could pass 2 instead of 2.0, but passing a very long int literal with more than 16 digits may fail at compile-time:

proc p(i, j, k: int; x, y: float; s: string) =
  echo s

var
  n: int16
  m: int # int64 would not compile
  z: float32
p(n, n, m, 1234567890, z, "")

Actually, float32 types and int literals up to ten digits seem to work for float parameters, but even on 64-bit systems, the int64 data type is not allowed for int parameters. As you see from the example above, it is possible to pass the same variable multiple times as a parameter, and empty string literals are of course allowed too.

Nim does also support default values for proc parameters and named parameters, that is, we can leave parameters unspecified and use the default value, or use the actual parameter names like in a variable assignment when we call a proc:

proc p(i: int; x: float; s: string = "") = echo i.float * x, s
p(x = 2.0, i = 3)

Here we used named parameters when calling the proc p(), this way we can freely order the parameters, and as parameter s has a default value, we can let it unspecified and just use the default value.

Functions always return a result, and procedures can return a result, but they don’t have to. In the C language, function results can just be ignored, but in Nim whenever there is a result, then we have to use it at the call site, that is, we have to assign the returned value to a variable, or we have to use it in an expression. Nim enforces this, as generally, the returned value is important, the returned value may be the actual result as in a sin() call, or it may give us additional information, like the number of read characters when we do text processing, or maybe an error indication, like the end of the file. For the rare conditions when we really intend to ignore the result of a function call, we can call that function as discard myProcWithResult(a, b,…​). Another solution is to apply the {.discardable.} pragma to the function definition, we will learn more about pragmas later. When a procedure should not return a result, then we can use the void return type or just leave the return type out — the latter is recommended, void types are used only rarely in Nim. When the proc has no parameters at all, then we can even leave out the empty parameter list in the proc definition:

proc p1() =
  echo "Hello and goodbye"

proc p2 =
  echo "Hello and goodbye"

proc p3: void =
  echo "Hello and goodbye"

Calling Procedures

When we call a procedure or a function, that is when we intend to execute it, then we have always to specify a parameter list enclosed in brackets, but the parameter list can be empty:

var i = myFunc(7)
var j = myF()
var p = myF # not a function call, but assignment of the proc to variable p

Note that the last line in the above code is not a call of myF(), but an assignment of that function to the variable p. Will will discuss this use case soon.

We have already learned, that we can also use the method call syntax, like 7.myFunc instead of myFunc(7), that we can use the command invocation syntax like in echo "Hello", and that we should avoid putting a space between the proc name and the opening bracket, as that would be interpreted as a command call with a tuple argument. When the function or proc expects multiple arguments, then we separate the arguments with commas, and we put generally a space after each comma:

proc p(i, j: int): int = i + j
echo p(1, 2) # ordinary proc call
echo 1.p(2) # method call syntax
echo p 1, 2 # command invocation syntax
echo p (1, 2) # argument looks like a tuple, so this would not compile

For the proc definition above, we wrote the body statement directly after the equal sign — that is possible, and sometimes used for very short procs. And indeed, here p() is a function.

In the examples above, we have passed plain integers as parameters to procedures, but of course, proc parameters can have any type, we can pass strings, arrays, objects, and all that. The way we pass the parameters to the procs is sometimes called "pass by value", an old term introduced for the Pascal language, used to indicate that the passed parameter seems to be copied to the proc, the proc is not able to modify the original instance. In the next section, we will learn about the var parameter type, which is used when we want to allow the proc to modify the original instance. In the wirthian languages, the procedure parameters actually get copied, so inside the proc, we could modify it, but we modified only the copy, and the original instance remained unchanged. In Nim, it is a bit different. When we pass parameters by value to a proc, we can not modify it at all in the proc body. When we require a mutable copy, we have to generate that copy ourselves in the proc body. This allows some optimizations: Nim needs not really to copy the proc parameters, as they are immutable, Nim can just work with pointers to the original instances internally. Actually, there are rumors, that for parameters smaller than 3 * sizeof(float) Nim copies the instances, but for larger instances, Nim works internally with pointers to the original value. But this is an implementation detail — data copied to the procs stack allow the fastest access, but on the other hand, the initial copy process can be expensive, so it is a compromise.

Procedure Parameters of Var Type

Our sqr() function above accepts only one parameter, and that parameter is a value type, which indicates that we can not modify it in the procedure body. That fact is useful to know for the caller of a proc, as one can be sure that the passed parameter is not modified and is available unchanged after the proc call.[34] But of course, there are situations where we may want that a passed parameter is modified. Let us assume that we want to "frame" a passed string, for example, we want to pass in the string "Hello" and want to change it to "* Hello *". Further, let us assume that we may sometimes want to use other characters instead of the asterisk, maybe a + sign.

proc frame(s: var string; c: char = '*') =
  var cs = newString(2)
  cs[0] = c
  cs[1] = ' '
  insert(s, cs)
  add(s, ' ')
  add(s, c)

# we can call that proc like
var message = "Hello World"
frame(message)
echo message

Note: In the wirthian languages, we actually put the var keywords for proc parameters in front of the parameter name, that is we would have to write proc frame(var s: string; c: char = '*') = for the procedure header.

The frame proc above accepts two parameters and returns no result. The first parameter has the type string, it is not a value parameter but a var parameter, which is indicated by the var keyword between the colon and the type of the parameter. Note that we use here again the keyword var that we used earlier to declare variables. The main reason that we use again the same keyword is that we do not want to use a new one — var proc parameters are different from var declarations. Parameters of var type can be modified in the procedure body, and that modification is visible after the proc call.[35] The second proc parameter is a plain value type, it is a character that has the default value '*'. To specify a default value for a parameter, we write an equal sign after the parameter type followed by the actual default value, as we would do in an assignment. Indeed, as in an assignment, we can even leave out the colon with the data type in this case, at least for the case that the compiler can infer the correct data type from the assigned default value. Default values are useful for parameters that have in most cases the same value but can be different sometimes. The advantage is, that when calling that proc, we can just leave that parameter out. For default values, we have to be a bit careful, only value parameters can have default values, and when we call a proc with many parameters with default values, it may not always be clear which parameter we pass and for which parameter we want a default value.

To generate the frame around the passed-in string, we have to insert two characters at the front of the string and append two more characters. Inserting in strings is not a very cheap operation, as it involves moving all following characters. So we try not to insert two single characters, but we first create a short string consisting of the passed c character and a white-space character and then insert that two-character string at the front of the passed string. We use the standard procedure newString() with parameter 2 to create a new string of length 2 with undefined content and then fill in the content by using the subscript operator. We could have used the add() proc to add that two characters to an empty string, but that is a bit slower. Then we use the standard procedure insert() to insert our two-character string at the front of our passed string. Finally, we add a whitespace and the c character to the passed string. The passed string is now modified, it is 4 characters longer. That modification is noticeable for the caller of that proc, that is, echo() will print the modified version. Actually, when we think about it, we may get the feeling that our strategy to first create the two characters string cs is a bad idea, as the allocation may cost more time than just inserting the individual characters directly.

Passing mutable arguments to procedures by use of the var keyword was sometimes called "pass by reference" in the old wirthian languages like Pascal. This leads to confusion for some people, unfortunately. Of course, proc var parameters are not really related to Nim’s ref type. Well, using Nim’s ref data types would also allow modifying proc arguments, in the same way as using pointers would do it. But we never use ref types in Nim just to be able to modify passed data in procs, and also not to avoid a possible expensive copy operation for value types. We could create a ref instance with var intRef: ref int = new int, pass that intRef to a proc and so allow modifying the actual value where the intRef points to from inside the proc. But that would be silly, as the var parameter is available for that. In Nim, we use reference types, when we really need them, e.g. when we really need reference semantics, or when we have to create highly dynamic, many to one data types, like tree structures.

When we call a proc or function with multiple arguments, then we have to pass the arguments in the same order as they are specified in the proc declaration.

Our frame() proc above modifies the passed string. We could have instead decided that the proc should not modify the string, but should return a new string consisting of the frame and the passed string in the center. Generally, when creating procs, we have to decide what is more useful — modifying a passed value or returning a modified copy. And sometimes we have to regard efficiency too. Returning newly created large data types like strings may be expensive. A string is not a trivial structure, as it contains the dynamic buffer for the string content, which has to be allocated. On the other hand, for the passed var string we inserted characters, which involves moving characters and is also not a really cheap operation, and maybe when we insert a lot, the string buffer must be even enlarged, which is again expensive. So for this use case, it is not really clear what approach is better — we used the var parameter mainly to introduce var parameters. OK, let us investigate how a function that returns a modified string may look:

func framed(s: string; c: char = '*'): string =
  var res = newStringOfCap(s.len + 4)
  add(res, c)
  add(res, ' ')
  add(res, s)
  add(res, ' ')
  add(res, c)
  return res

# we can call that proc like
echo framed("Hello World")
echo framed("Hello World", '#')

The above code is one possible solution. We can use the keyword func instead of proc here, as we only return a result but modify no states. We pass the initial string and the character for the frame both as plain value parameters and return a newly created framed string. In the function body, we start with an optimized version of the procedure newString() from the system module, called newStringOfCap(). Like newString() that procedure creates an empty string variable, but it ensures that the data buffer of the new string has exactly the specified size. That is an optimization, which makes sense in our use case, as we know that our newly created string will have 4 characters more than the passed string. So we can avoid that the result string has to be enlarged while we add characters or the initial string, and we ensure at the same time that no space is wasted — the data buffer size of the new string will be a perfect fit for the desired result. The rest of the function body is not really interesting, we just add() what is needed and return the result. Well, earlier we said that add() is not extremely fast. So when you have to frame millions of strings each day, you may consider avoiding add(), and you know already enough about Nim to do it. Just try it. You may start with a string of the right size containing undefined content created by newString(s.len + 4), and then you may copy in the required data in a loop character for character. Or you may use the slice operator to insert the passed string into the new string.

Click to see a possible solution
func framed(s: string; c: char = '*'): string =
  var res = newString(s.len + 4)
  res[0] = c
  res[1] = ' '
  res[2 .. s.high + 2] = s # we may insert the string by using the slice operator or
  # for p in 0 .. s.high: # we can use a for loop and
  #   res[p + 2] = s[p] # the subscript operator
  res[^2] = ' '
  res[^1] = c
  return res

The situation, where we may need a procedure that works on a var parameter in one case and returns a modified copy in another case, is not that rare. So for example, Nim’s standard library contains a procedure called sort(), which can sort container data types in place, and a procedure called sorted(), which returns a sorted copy. This code duplication is not really that nice. Of course, sorted() is the more universal solution, as we can always replace sort(data) with data = sorted(data). But the latter creates a temporary copy, which may not be optimal for performance. Since Nim version 1.2 a dup() macro is available from the sugar module, which creates copies of variables and then applies one or multiple in place procs on the copy. So the procs sorted() or our proc framed() would be unnecessary. We can use dup() as in this example:

from sugar import dup

proc frame(s: var string; c: char = '*') =
  var cs = newString(2)
  cs[0] = c
  cs[1] = ' '
  insert(s, cs)
  add(s, ' ')
  add(s, c)

echo "Hello World".dup(frame)
echo "Hello World".dup(frame, frame)
echo "Hello World".dup(frame('#'))

Note that we apply frame() two times in the line before the last one — in the same way, we could apply a sequence of different procs. The result of the above program is

* Hello World *
* * Hello World * *
# Hello World #

Returning from a Procedure and the implicit Result variable

The execution of a procedure terminates when the last statement of the procedure body has been processed. We can also terminate a procedure earlier when we specify a return statement somewhere.

Functions and procedures which return a result can also terminate with the last expression of the procedure body, or earlier with a return expression like return i * i. Functions and procedures with a result declare automatically a mutable result variable for us, which is of the function’s return type and which we may use or just ignore. So for our previous sqr() function, we have various ways to write it:

func sqr1(i: int): int =
  i * i

func sqr2(i: int): int =
  result = i * i

func sqr3(i: int): int =
  return i * i

For short and simple procedures, the first form is often used. For longer procedures, where the result is constructed in multiple steps, like some string operations, using the result variable makes sense. And finally, when there exist multiple points where we may jump back, using return statements may make sense. One use case is an early error check, maybe we want to return -1 as some form of error indication when we write a procedure that should calculate the square root of an integer value. (Well in Nim we have other and sometimes better ways to catch errors, we will learn about that later.)

Generally, we should avoid writing something like

func sqr(i: int): int =
  result = i
  i * i

as it may be unclear in this case if the expression i * i is returned or the result variable with the value i. For Nim v1.6 we will get a warning or an error message in such a case.

For the performance of our code, it may have a tiny benefit to only use the result variable and fully avoid return statements, as in this case for a function call like var i = sqr(j) the result variable may be just an alias for the actual result i here, so that the compiler can optimize the code and avoid temporary copies. But that are rumors and may depend on the actual compiler version.

Var Return Type

A procedure, converter, or iterator may return a var type, which can be modified by the caller. The Nim compiler manual provides this basic example:

var g = 0
proc writeAccessToG(): var int =
  result = g
writeAccessToG() = 6
assert g == 6

This way, we can call a proc and immediately assign a new value to the result. In the above example, this works, as the result is an alias for the global variable g.

Actually used are var return types for iterators like mitems() or mpairs(), which allows modifying the yielded results. For details and restrictions of the var return type, you should consult the Nim compiler manual:

References:

Proc name overloading

Note that we used the proc names sqr1, sqr2, and sqr3 above. Using the same name with the same argument types multiple times would result in a redefinition error, as the compiler could not know what proc body should be executed when that proc name is called.

But Nim supports so-called proc overloading, that is we can use the same name, when the parameter list is different, as the compiler can select from the parameters in the proc call which proc has to be called:

func sqr(i: real): real =
  i * i

We have only changed the parameter and result data type. Now there is no conflict with the proc with the same name which, we defined for integers. Note that Nim uses only the parameter list for overload resolution, but not the result type of a proc or function. The reason for that is, that Nim supports type inference, and that would not work when we would have two procs with the same name each accepting an int parameter, but one returning an int and one returning a float number.

Nim does also support named arguments in proc calls, that is, we could invoke the proc above with sqr(i = 2.0). Named arguments can be useful when procs or functions have many arguments, maybe some with default values, and we do not remember the order of parameters or when we want to specify only a few.

Objects and Ref Objects as Procedure Parameters

In the previous section, we learned that we have to pass var parameters when the procedure should be able to mutate the variable permanently. This is also valid when the parameters are objects. When a procedure should modify fields of an object parameter, then we have to pass that object as a var parameter. In the following example, proc t1 gives a compiler error because that procedure tries to modify a field of an object while the object instance is not passed as a var parameter. If we remove proc t1, then we can compile and run the example:

type O = object
  i: int

proc t1(o: O) =
  o.i = 7 # Error: 'o.i' cannot be assigned to

proc t2(o: var O) =
  o.i = 13

proc main =
  var x = O(i: 3)
  echo x.repr
  t2(x)
  echo x.repr

main()

The output is:

[i = 3]
[i = 13]

Proc t2 gets a var parameter and can modify fields of the passed object. Here we used the expression echo x.repr to print the whole object. Strings and sequences are value objects in Nim, so you have to pass them as var parameters when you want to change their length or when you want to modify elements. This code would give you compile errors unless you add the var keyword to make the proc parameters mutable:

proc t1(s: string) =
  s.setLen(7)
  s[0] = 'x'

proc t2(s: seq[int]) =
  s.setLen(7)
  s[0] = 13

This was not really surprising. But what when we use a reference to an object and pass that to procedures as value and as var parameters? In the code below, proc t1 gets a variable of type ref object and the procedure can modify fields of the passed instance. That can be indeed surprising. In this case, passing the ref object without the use of the var keyword means only that we can not mutate the ref value itself in the procedure, but we are allowed to modify the fields of the object. For proc t2, we pass a var parameter. As always, we can modify a var parameter in the procedure, so we can assign it a newly created instance.

type O = ref object
  i: int

proc t1(o: O) =
  o.i = 7

proc t2(o: var O) =
  o = O(i : 11)

proc main =
  var x = O(i : 3)
  echo x.repr
  t1(x)
  echo x.repr
  t2(x)
  echo x.repr

main()

When we compile and run the above code, we get:

ref 0x7f054a904050 --> [i = 3]

ref 0x7f054a904050 --> [i = 7]

ref 0x7f054a904070 --> [i = 11]

For a ref object, the repr() function gives us the address of the object instance in memory and the contents of its fields. The first two echo() statements show the same address, indicating that proc t1 has modified only a field of our instance, the instance itself (its address in memory) was not changed. But proc t2 has created a new instance and assigned that value to the variable x in the main() procedure. We notice this as the address of variable x has changed. The old instance variable with address 0x7f054a904050 is now unused and will be freed by the Nim memory management.

Special Argument Types: OpenArray and Varargs

The openArray and varargs data types can be used only in parameter lists. OpenArray is a type that allows passing arrays and sequences to the procedure or function. Allowing that makes sense, as arrays, as well as sequences, store their content in a block of memory, which can be processed uniformly. Although arrays generally do not have to start with index number 0, when passed as openArray, the first element is mapped to index 0, and the index of the last element is available by using the high() function on the passed array parameter. Whenever we write a procedure that accepts an array or a sequence, we should consider using the openArray parameter type to allow passing in both data types. Strings can also be passed to procs accepting openArrays with char base type. Note that a proc with openArray parameter type can not change the length of a passed seq, as for the openArray parameter type sequences are handled like arrays. So in the code below, proc t1 generates a compiler error while t2 compiles and works fine.

proc t1(x: var openarray[int]) =
  x.setLen(7)

proc t2(x: var seq[int]) =
  x.setLen(7)

Actually, since Nim version 1.6, it is possible to use the openArray type as the result type of procs and even as local variables. But these View types are still experimental, see https://nim-lang.org/docs/manual_experimental.html#view-types.

The varargs parameter type is similar to the openArray type, but it additionally allows passing an arbitrary number of single arguments. The compiler automatically collects the single arguments into an array for us, so in the proc body we can use it like an array, e.g. iterating over it.

proc print(s: varargs[string]) =
  for el in s:
    stdout.write(el)
    stdout.write(", ")
  stdout.write('\n')

print("Hello", "World") # compiler builds the array for us
print(["Hello", "World"]) # we generate the array our self

There exists a variant of the varargs argument type that performs a type conversion automatically by applying a proc on all arguments. For example, varargs[string, $] would apply the stringify operation on the passed arguments automatically. That is what echo() does.

Varargs arguments may be only allowed for the last argument in a parameter list.

Finally, we may wonder if it makes sense to specify a parameter of type var varargs. If we try to pass a constant string this will obviously not work, and if the compiler generates an array for us, it does also not work, the automatically generated array seems to behave like a constant array. But may we pass an array variable? Let us try:

proc print(s: var varargs[string]) =
  s[0] = "Goodbye"
  for el in s:
    stdout.write(el)
    stdout.write(", ")
  stdout.write('\n')

var msg = ["Hello", "World"]
print(msg)

Surprisingly, that does not compile, while it works when we replace varargs with openArray.

Procedures bound to a Data Type

In some other programming languages like Python or Ruby, we can define class methods or static methods, which are bound to a class or type and can be called as MyType.myProc. In Nim, we can do something similar by use of the typedesc proc parameter type:

type
  Factory = object
    name: string

proc start(t: typedesc[Factory]) =
  echo "Factory.start"

Factory.start

Here, we used the method call syntax instead of start(Factory). We will learn more about the typedesc data type later.

Scoping, Visibility, and Locality

Scoping, visibility, and locality are important concepts in computer programming, to keep the source code clean. Imagine that a variable that we declare at some point in our program would be visible everywhere. That would even for medium size programs generate a lot of confusion — whenever we would need a variable, we would have to carefully check which names are already in use. And for performance, it would be bad also, as all variables declared somewhere would reside permanently in memory.

So most programming languages including Nim support the concept of locality — names declared inside a procedure body or inside another form of a block are only visible there and can only be used there. We say that they are only visible in that scope. For Nim, we can say that whenever Nim’s syntax requires a new level of indentation, that is a new statement block, then all symbols declared in that block are only visible in that block and in sub-blocks of this block, but not outside that block. Nim has another important concept of visibility, which is called modules and allows separation of our code in logically separated text files with well-defined visibility rules, we will discuss modules later.

Visibility is really a simple concept, let us regard this useless example:

var e: float = 2.7

proc p1 =
  var x: float = 3.1415
  if x > 1.0:
    var y = 2.0 * x
    echo y # OK
  echo x # OK
  echo y # compile error, y is not visible
  echo e # OK, e is declared globally, so it is visible everywhere

echo e # OK
echo x # ?
echo y # ?

In line one, we declare a so-called global variable, that one is visible after the declaration, that is below the line where it is declared, in the entire program. The variables declared in the proc p1 are called local variables, they are not visible outside of that proc p1. The variable x is declared at the start of the proc body and is visible in the whole proc everywhere, while variable y is declared in the if block and is visible only there. So it should be clear if the last two echo statements for x and y compile fine? Remember that symbols that we define inside a new scope may shadow symbols that were visible outside the actual block, e.g. by defining a variable named e of arbitrary type in the proc p1 from above would shadow the global variable e, that is the global variable e would become invisible until execution of proc p1 terminates. We discussed shadowing already in the introducing section Scopes, Visibility, Locality, and Shadowing.

Related to the visibility of variables is their lifetime, that is the duration of how long they exist and how long they can store a value. Global variables exist for the entire program runtime — when you have assigned a value to it that value can be used everywhere as long as the program runs, and as long as you do not assign a different value, of course. Global variables are generally stored in a special memory region, which is called the BSS region.

Variables of value type defined locally inside a procedure or function do exist only for the execution time of that proc, that is, they are created when the proc is invoked and vanish when the proc terminates, that is when execution continues with the statement following on the proc call.

Local variables declared in a proc reside in a special memory region of the RAM, which is called the stack. The stack is nothing more than an arbitrary part of the hole RAM that is used in some clever fashion: The memory words in it are used in consecutive order. A so-called stack pointer is used to indicate the address of the first free area in that stack. So when a proc is called, which may have n bytes of local variables, then the compiler can use the area where the stack pointer points to for that variables, and when the proc is called then the stack pointer is increased by that size. So the stack pointer points again to the next free area of the stack, and another proc can be called in the same way from within the current proc. Whenever a proc terminates, the stack pointer is set back to the value that it had when the proc started execution. This method of memory management is simple and fast, but it does only work when the total amount of memory that the local variables in a proc needs is known at compile-time, so that the compiler can adjust the stack pointer accordingly. It does not work for dynamically sized data types like strings or sequences.

Note that pointers and references are value types themselves, we can regard pointers and references as a plain integer variable interpreted in a special way — as a memory location. But the memory blocks to which the pointers and references may point and that are allocated by alloc() or new() is different: That memory blocks are not allocated on the stack, but in the ordinary RAM which we call heap to separate it from the stack.

So why can the stack not be used for memory blocks that alloc() or new() provides for us: An important fact for the use of the stack to store variables is that the total size, which is needed by a proc for all the static variables, must be a compile-time constant. The stack pointer is adjusted by that amount when the proc starts and all the local variables are accessed with a fixed offset to that stack pointer then. When we use alloc() or new() in a proc, then we may call that multiple times as we did in our previous list example, and for alloc() an additional fact is that the byte size that alloc() should reserve can be a runtime value. So the total amount of RAM that alloc() or new() would allocate is a runtime value, and we can not use the stack for it. Instead, alloc() and new() allocates a block of memory in a more dynamic fashion, which is basically that they ask the OS for a free block of the right size somewhere in the available RAM. That block is later given back to the OS for reuse by functions like dealloc() or automatically by the GC.

Let us at the end of this section investigate some special cases:

While in languages like C we have always a well-defined main() function and all program code is contained in this function or in other functions which are called from this main function, in Nim we have also global code as in scripting languages Ruby or Python:

var i: int
while i < 100:
  var j: int
  j = i * i
  echo j
  inc(i)

It should be clear that the global variable i resides in the BSS segment. But what is with the variable j declared in the body of the while statement? It is clear that that variable is only visible inside the body of the while statement. But does j reside on the stack? There seems to be no proc involved, so there may be no stack? The variable j may reside in the BSS segment too? That is not really clear and may be different for different Nim compilers, maybe. But why should we care for that detail at all? Well, it may be important for performance. Local proc variables allocated on the stack are generally optimal for performance, and they are optimized by the compiler very well. We will learn more about the reasons for that later when we discuss the data cache. For now, we should only remember that it may be a good idea to avoid global code and put all code in procs. We may have an arbitrary named main() proc then and call that from global scope only. At least for the current Nim v1.6 that seems to be a good idea, potentially later versions or other implementations will automatically move all global code into a hidden proc for us.

For optimal performance, put all your code in procedures or functions and avoid global code and when possible global variables.

Let us discuss the above while loop again, but this time in the body of a proc:

proc p =
  var i: int
  while i < 100:
    let j: int = i * i
    echo j
    inc(i)

When we carefully investigate that proc within the while loop, we may wonder about two points. First, we said earlier that we can and should use the let keyword instead of var when there is only one assignment to a variable, so the variable can be regarded as immutable. But the loop is executed 100 times, so how can we say there is only a single assignment to variable j? The trick is, that j is local to the while loop, and that j is virtually newly created and initialized to 0 for each iteration. So let is OK and the compiler does not complain.

We can test that fact with this simple program:

proc main =
  var i: int
  while i < 10:
    var a: int
    a = a + 1
    echo a
    inc(i)
main()

The output is 1 for each loop iteration, as variable a is virtually newly created for each loop iteration.

We said virtually newly created because we can not be sure how the compiler may handle it internally. Is storage for variable a already allocated when the proc is invoked, that is, in the same way as storage for the loop counter variable i is allocated on the stack when the proc is called. Or is storage for variable a reserved for each loop iteration by increasing the stack pointer at the start of the loop and resetting it at the end of the loop. We can not be sure without reading the compiler source code, but finally, we should not care, as it does not really matter.

Generics

In the previous section, we defined an sqr() proc for ints and one for float numbers. Both procs look nearly identical, only the data types differ. For that case, we can use so-called generic procedures.

func sqr[T](v: T): T =
  var p: T
  p = v * v
  return p

echo sqr(2)
echo sqr(3.1415)

We put a square bracket after the function name, which includes a symbolic name, and that name is then used instead of concrete types in the proc header or in the proc body.

We can now call that proc with parameters of different types, including int and float types. You may wonder why that works — Nim is a statically typed language, so how can the parameter of function sqr() as well accept an integer as a floating-point number? Is there a hidden type conversion involved? No, the trick is that whenever we call that generic proc with a different type, then a new proc or function is instantiated. As we called the generic sqr() proc with an int and a float parameter, during compile time the compiler creates machine code for two separate functions, one which is called when an int is passed as a parameter, and one which is called when a float is passed. If we called that proc name again with an int or float parameter, then one of the two existing procs would be used. But for a different, still unused data type like float32, again a new proc would be instantiated. In this way, generics procs can lead to some code bloat. Note that calling the generic function with a data type like a character or a string would fail, as these types do not support multiplication with themselves.

A slightly different notation is available by so-called or types:

func sqr(v: int or float): auto =
  var p: typeof(v)
  p = v * v
  return p

echo sqr(2)
echo sqr(3.1415)

Here, we have limited the parameter types to the int type or the float type. We could have defined also a custom type first, like type MyNum = int or float, and used that type for the type of our sqr() proc. These or types are also called type classes. Instead of keyword or the | character can be used for defining type classes. Again, the compiler would instantiate two separate functions for both data types. As we had not the symbolic type T available here, we have used the keyword auto as the return type, and for the type of variable p we used the macro typeof(). The type auto for the return type works as long as the function returns a well-defined type. Note that we can not decide at runtime what type the function should return, so a construct like if cond: return 2 else: return 3.1415 would not work, at least not when the values are variables of different types. For the literal value, it may work, as the compiler may be smart and guess that we want to return the float literal 2.0.

A bit of care is needed when we define procs for mutable or types:

# proc t(s: var seq[uint8] | var seq[char]) =
proc t(s: var (seq[uint8] | seq[char])) =

Here we try to define a proc called t which should accept a mutable seq[uint8] or a mutable seq[char] as a parameter. While the first line compiles fine, the seq[char] would be immutable. The correct notation is shown in the second line. This behavior was labeled "won’t fix" in GitHub issue tracker, so we have to remember this case, see https://github.com/nim-lang/Nim/issues/15063#issue-665553657.

Let us assume that you want to define a proc that accepts two numbers of type int or float and that returns a float. You may write it in one of these ways:

proc sqrsum(x, y: int | float): float =
  (x * x).float + (y * y).float

proc sqrsum2[T](x, y: T): float =
  (x * x).float + (y * y).float

proc sqrsum3[T1, T2](x: T1; y: T2): float =
  (x * x).float + (y * y).float

var i: int = 2
var x: float = 3.0

echo sqrsum(i, x)
#echo sqrsum2(i, x)
echo sqrsum2(x, 2)
#echo sqrsum2(2, x)
echo sqrsum3(i, x)

The commented-out lines would give you a compiler error. The reason for this is, that the proc sqrsum2[T] defines a generic proc, but the compiler enforces that both parameters have the same type. The expression sqrsum2(x, 2) compiles fine, as due to the first parameter x the compiler instantiates a proc for a float parameter type, and then converts the second parameter, which is an integer literal, to float automatically. This automatic conversion is only done for literal numbers, not for variables. The expression sqrsum2(2, x) does not compile, as due to the first parameter, which is an integer literal, a proc for integer parameters is instantiated, and the second x parameter of float type is not compatible with the instantiated proc.

Generics can become a bit complicated, as we may use multiple different generic types for different proc parameters. And we can use generics also for object types, we may for example create lists as we did for our names list that works not only for strings, but can work with other data types like numbers or sequences in a very similar way. We may explain that in more detail later.

Example for the use of Generics

Generics are used a lot in Nim’s standard library. Most container types like sequences or tables accept generic types, and generic procedures like sort() are provided which can easily sort arbitrary data types and objects. We have only to provide a cmp() proc for our user-defined data types, which sort() can call to compare the values during the sorting process.

We will demonstrate the use of generics for library modules with a few tiny examples: Assume we create a library that should be able to store and process arbitrary data types. The stored values may have well-defined relations, which enables ordering or much more complicated spatial relations. Triangulation of spatial data points or grouping of the data in structures like RTrees for fast point location, as well as geometric processing with algorithms like finding the convex hull are some examples. To make our example simple and compact, we define a generic container type that can store only two values of an arbitrary data type. The container allows for the sorting of the elements by size. The following code example defines a generic container called MyGenericContainer, a proc to add() data objects into the container instance and a sortBySize() proc to sort the two elements:

type
  MyGenericContainer[T] = object
    storage: array[2, T]

proc add[T](c: var MyGenericContainer[T]; x, y: T) =
  c.storage[0] = x
  c.storage[1] = y

# sort by direct field access
proc sortBySize[T](c: var MyGenericContainer[T]) =
  if c.storage[0].size > c.storage[1].size:
    swap(c.storage[0], c.storage[1])

# a simple stringify proc for our container data type
proc `$`[T](c: MyGenericContainer[T]): string =
  `$`(c.storage[0]) & ", " & `$`(c.storage[1])

type
  TestObj1 = object
    name: string
    size: int

proc main =
  var c: MyGenericContainer[TestObj1]
  var a = TestObj1(name: "Alice", size: 162)
  var b = TestObj1(name: "Bob", size: 184)

  add(c, b, a)
  echo c
  c.sortBySize
  echo c

main()

The sortBySize() proc of the above examples accesses the size field of our data objects directly, so we can use the container for arbitrary data types as long as the data types have a size field and as long as a > proc is defined for the data type of the size field. In the above example, we have defined a $ procedure to convert instances of our container to a string, which allows us to call the echo() function on it. The output of our program looks like

(name: "Bob", size: 184), (name: "Alice", size: 162)
(name: "Alice", size: 162), (name: "Bob", size: 184)

We can avoid the restriction of a matching field name when we provide getter and setter procedures which the library procs can use to access the important fields:

type
  MyGenericContainer[T] = object
    storage: array[2, T]

proc add[T](c: var MyGenericContainer[T]; x, y: T) =
  c.storage[0] = x
  c.storage[1] = y

proc sortBySize[T](c: var MyGenericContainer[T]) =
  if c.storage[0].size > c.storage[1].size:
    swap(c.storage[0], c.storage[1])

proc `$`[T](c: MyGenericContainer[T]): string =
  `$`(c.storage[0]) & ", " & `$`(c.storage[1])

type
  TestObj1 = object # arbitrary field names
    name: string
    length: int

# this getter proc enables sorting
proc size(t: TestObj1): int =
  t.length

proc main =
  var c: MyGenericContainer[TestObj1]
  var a = TestObj1(name: "Alice", length: 162)
  var b = TestObj1(name: "Bob", length: 184)

  add(c, b, a)
  echo c
  c.sortBySize
  echo c

main()

In the example above, our TestObj1 data type has no field with a name matching the sortBySize() proc, but we define a size() proc for our data type which that library function can use. This solution is more flexible, and when we add the inline pragma to the used size() proc or when we compile with link-time optimization (LTO) enabled, then the overhead should be negligible. The two examples above are located in a single file each, but of course, for practical use, we would use separate modules for the library and the application part as in

#module t3.nim
type
  MyGenericContainer*[T] = object
    storage: array[2, T]

proc add*[T](c: var MyGenericContainer[T]; x, y: T) =
  c.storage[0] = x
  c.storage[1] = y

proc sortBySize*[T](c: var MyGenericContainer[T]) =
  if c.storage[0].size > c.storage[1].size:
    swap(c.storage[0], c.storage[1])

proc `$`*[T](c: MyGenericContainer[T]): string =
  `$`(c.storage[0]) & ", " & `$`(c.storage[1])
import t3

type
  TestObj1 = object # arbitrary field names
    name: string
    length: int

proc size(t: TestObj1): int =
  t.length

proc main =
  var c: MyGenericContainer[TestObj1]
  var a = TestObj1(name: "Alice", length: 162)
  var b = TestObj1(name: "Bob", length: 184)

  add(c, b, a)
  echo c
  c.sortBySize
  echo c

main()

The example with direct field access would look for different modules like this:

# module t4.nim
type
  MyGenericContainer*[T] = object
    storage: array[2, T]

proc add*[T](c: var MyGenericContainer[T]; x, y: T) =
  c.storage[0] = x
  c.storage[1] = y

proc sortBySize*[T](c: var MyGenericContainer[T]) =
  if c.storage[0].size > c.storage[1].size:
    swap(c.storage[0], c.storage[1])

proc `$`*[T](c: MyGenericContainer[T]): string =
  `$`(c.storage[0]) & ", " & `$`(c.storage[1])
import t4

type
  TestObj1 = object
    name: string
    size: int

proc main =
  var c: MyGenericContainer[TestObj1]
  var a = TestObj1(name: "Alice", size: 162)
  var b = TestObj1(name: "Bob", size: 184)

  add(c, b, a)
  echo c
  c.sortBySize
  echo c

main()

You may wonder why we do not have to export the size field of our TestObj1 (or maybe the object itself also) as it is used from code defined in a different module. The reason why we do not need export markers is that the sortBySize() is defined in the library module, but as it is a generic procedure, it is instantiated and executed in the application module. For the same reason, we had not to export the size() getter procedure before.

Finally, one more way to use generic library modules is by passing procedure variables to the library functions. The passed in procedures may provide access to properties or attributes of the stored objects, or they may offer relations between the objects. The latter is often used for sorting purposes:

# module tx.nim
type
  MyGenericContainer*[T] = object
    storage: array[2, T]

proc add*[T](c: var MyGenericContainer[T]; x, y: T) =
  c.storage[0] = x
  c.storage[1] = y

proc sortBy*[T](c: var MyGenericContainer[T]; smaller: proc(a, b: T): bool) =
  if smaller(c.storage[1], c.storage[0]):
    swap(c.storage[0], c.storage[1])

proc `$`*[T](c: MyGenericContainer[T]): string =
  `$`(c.storage[0]) & ", " & `$`(c.storage[1])
import tx

type
  TestObj1 = object
    name: string
    size: int

proc smaller(a, b: TestObj1): bool =
  a.size < b.size

proc main =
  var c: MyGenericContainer[TestObj1]
  var a = TestObj1(name: "Alice", size: 162)
  var b = TestObj1(name: "Bob", size: 184)

  add(c, b, a)
  echo c
  c.sortBy(smaller)
  echo c

main()

Here, we have modified the sort() proc of our library module in a way that it takes an additional procedure parameter. In this case, we use a procedure signature that takes two object instances and returns a boolean value indicating if the first parameter is smaller than the second. In our application module, we define a matching procedure and pass that one to the sortBy() procedure. Again we get the desired sorted output:

(name: "Bob", size: 184), (name: "Alice", size: 162)
(name: "Alice", size: 162), (name: "Bob", size: 184)

This last method is often used in Nim’s standard library, e.g. for sorting sequences with custom objects. Unfortunately, this way can introduce some performance regression, as the procedure variable has to be passed to the called proc, and so inlining of that passed proc is not possible for the compiler. [36]

Method Call Syntax

A useful coding style introduced by OOP languages is the method call syntax, which was initially used in the OOP programming style for objects, and later applied by languages like Ruby to all data types. Ruby in some way regards all data as objects.

Method call syntax means, that for example, for a variable s of data type string we do write s.add(c) instead of add(s, c). Or for an integer variable i, we may write i.abs instead of abs(i). That is, we put the first parameter of the proc parameter list in front of the proc name, and separate that parameter from the proc name by a period. The Nim compiler regards both notations as equivalent. The advantage of the method call syntax is, that we may save a character and that it is more clear with what "object" we are working, as it stands in front of the expression.

Most OOP languages allow that notation only for a class, for example, the string class may declare all possible operations that can be done with strings, and the method call syntax is used for that operations. One problem is, that it can be difficult to add more operations that can be used in that style, as often all that operations are defined in the class scope. Ruby fixed that restriction by allowing so-called reopening of classes, that is, user can later add more operations.

Nim simple allows that notation generally, as did the D language, but D used the term Uniform Function Call Syntax (UFCS) for it.

Procedure Variables

Procedures and functions are not always fully static entities. We can assign procedures and functions to variables, and we can pass whole procedures or functions as parameters to other procedures or functions. And functions can even generate and return new functions. Let us investigate how procedure variables work:

var
  p: proc(i: int): int

proc p1(i: int): int =
  i + i

proc p2(i: int): int =
  i * i

p = p1
echo p(7)
p = p2
echo p(7)

The output of the two echo statements should be 14 and 49 — we called in both cases the same proc variable with the same parameter, but the proc variable p was in the first call an alias for p1, and in the second call an alias for p2. Note, when we assign a proc to a proc variable, we only write the name of that proc, there is no () involved. That is because we assign that proc to the proc variable, but we do not call the proc in this case. Of course, when we assign a proc to a proc variable, then the proc signatures have to match, that is the parameter list and the result has to be compatible.

Now we use a function as a proc argument.

type
  EchoProc = proc (x: float)

proc t(ep: EchoProc; x: float) =
  echo "The value is"
  ep(x)

proc ep1(x: float) =
  echo "==> ", x

proc ep2(x: float) =
  echo x

t(ep1, 3.1415)
t(ep2, 3.1415)

A common use case for a function as a proc parameter is sorting. We can use the same sort procedure for different data types when we provide a cmp() proc that can compare that data type.

from algorithm import sort

proc cmp(a, b: int): int =
  if a < b:
    -1
  elif a == b:
    0
  else:
    1

proc main =
  var a = [2, 3, 1]
  a.sort(cmp)
  for i in a:
    echo i

main()

The sort() procedure is provided by the algorithm module. The sort() proc accepts an array or a sequence, and a cmp() proc that gets two parameters of the same type as the elements in the passed array, and that returns -1, 0, or 1 as the result of the comparison. We could easily sort other data types like strings or our custom objects by an arbitrary key, as long as we can provide a matching cmp() proc. For the cmp() proc it is important that it returns a well-defined result based on the input, and when both parameters are equal, it should really return 0. If you exchanged the return values 1 and -1 in the cmp() proc above, you would invert the sort order.

Nested Procedures and Closures

While in C, all functions must be defined in top-level scope, and nesting of functions is not allowed, in Nim procedures can contain other procedures. A special case occurs when the sub-procedures do access variables of the outer scope. In this case, the sub-procedure is called a closure:

proc digitScanner(s: string) =

  var pos = 0
  proc nextDigit: char =
    while pos < s.len and s[pos] notin {'0' .. '9'}:
      inc(pos)
    if pos == s.len:
      return '\x0'
    result = s[pos]
    inc(pos)

  var c: char
  while true:
    c = nextDigit()
    if c == '\x0':
      break
    stdout.write(c)
  stdout.write('\n')

digitScanner("ad5f2eo73q9st")

When you run this program, the output should be

52739

This program is not that easy, but when you think about it a bit, you should be able to understand it. The task is to extract from a string all the digits and to ignore the other characters.

To get the digits, we use a local procedure that uses the pos variable of the enclosing procedure and also access the parameter s of the enclosing procedure. The closure nextDigit() checks if the position in the string is still valid, that is, if it is still smaller than the length of the string, and also checks if the current character is a digit. The first check uses the standard procedure len(), which returns the length of a passed string parameter, that is, how many characters the string contains. We have used the method call syntax here instead of using the ordinary proc call len(s). The next check tests, if the current character is not a decimal digit. For that test we could use a series of compares like if c == '0' or c == '1' or …​ or c == '9'. But to make such tests easier and faster, Nim offers one more data type, the set type. And the notin operator tests, if a value is not contained in a set constant. An important point for the expression after the while statement is, that it is processed from left to right. That fact is here critical because we have first to check if pos is still a valid position before we can use the subscript operator [] to access the current character and test if it is not contained in the set. If the check for the valid position would not come first, then we may access an invalid position in the string, and we would get a runtime range error.

While the position is still valid, but the current character is not a digit, we increase the position. The while loop can end by two conditions: Either the current character is a digit, or we have reached the end of the string, and we have to stop. For the last case, we use a special stop mark, we return a special character which we have entered in escape notation as '\x0'. That is a very special character, that is used in C to mark the end of strings. It is the first character in the ASCII table and has the decimal value 0. We said earlier, that characters are encoded in 8 bit and correspond to the unsigned integer numbers 0 up to 255. '\x0' is just a special notation for the first character, which corresponds to the integer value 0. Well, when the end of the string is reached, then we return that character. Otherwise, we return the current character. Remember, from the while condition we know that the string end is reached, or the current character is a digit. As we tested for the string end before, we can only have the case that the current character is a digit now. But can we return that character immediately now? If we would, s[pos] would be a digit, and we would get exactly the same character for the next proc call! So we have to move to the next character by increasing pos before we return that character. For this, the pre-declared result variable is useful. We assign the current character to the result variable and then increase pos. As the last statement in our proc is not an expression but a plain inc() statement, the content of the result variable is returned. The other while loop in the outer procedure is very simple, we just call the closure in the body of the while loop and terminate the loop when we get the special Null character.

And finally, an example where one proc returns another procedure:

proc addN(n: int): auto = (proc(x: int): int = x + n)

let add2 = addN(2)
echo add2(7)

The output of echo() would be 9 in this case. This construct is sometimes named currying.

Anonymous Procedures

In the section Module sequtils in part III of the book, we will introduce a few functions which are often used in the functional programming style like map() or filter(). These functions get procedures as arguments that determine how container data types are converted. We can pass a regular named procedure as a second argument to procs like map() and filter, or in simple cases, we can just pass an anonymous proc or use the operator provided by the sugar module:

import sequtils, sugar

proc primeFilter(x: int): bool =
  x in {3, 5, 7, 13}

var s = (0 .. 9).toSeq # @[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

echo s.filter(primeFilter) # @[3, 5, 7]
echo s.filter(proc(x: int): bool = (x and 1) == 0) # @[0, 2, 4, 6, 8]

echo s.map(proc(x: int): int = x * x) # always @[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
echo s.map(x => x * x) # from sugar module

Here we use the toSeq() template to create our initial sequence with numbers 0 up to 9 — just to have not to type all the numbers in, we will explain templates soon. Then we apply the filter() proc to that sequence. The filter() proc expects as a second argument a function with an argument of the seq’s base type, returning a boolean value. We can pass the named function primeFilter(), or we can just pass an anonymous proc explicitly.

In the last two lines of our example, we use the map() function to convert the data of our sequence. Map() expects as a second argument a proc with a parameter of the seq’s base type, returning a result of the same type. In the line before the last one, we specify an anonymous proc as a parameter, while in the last line we use the operator from the sugar module to just specify the actual conversion.

compile-time proc execution

When a function is called only with constant arguments, then the compiler can execute it already at compile time:

func genSep(l: int): string =
  debugecho "Generating separator string"
  for i in 1 .. l:
    result.add('=')

const Sep = genSep(80) # function is executed at compile-time

echo Sep

Here, we used a function called genSep() to create a string constant at compile time. When we compile the above program, we get the message "Generating separator string". As that proc is not executed at program runtime, it is not included in the final executable program. Here we had to use the debugEcho() proc instead of the ordinary echo(), because echo() is not really a pure function, and the compiler would complain when we use echo() in a pure function. DebugEcho() is not really pure as well, but the compiler ignores that fact, which is OK for debugging purposes. We could even make genSep() a plain proc and then use echo(), the compiler would not complain. But it would complain, if, for instance, we would access global variables from inside the genSep() proc.

Inlining Procedures

Calling procedures and functions is always some overhead — proc parameters may have to be put on the stack or loaded into CPU registers, some CPU or FPU registers may have to be saved, the stack pointer and the program counter have to be updated, and finally the instruction cache has to be filled with new instructions.

So for tiny procs, actually calling the proc may take more time than processing the actual code inside the proc. To avoid this additional effort, procedures and functions can be inlined. The compiler may do this automatically for us, but we can support it by applying the {.inline.} pragma to tiny procs.[37] For inlined procs, the code is just inserted directly at the call site. This may increase the total executable size when the proc is used often. So we should use the inline pragma with some care. Another option is to just compile the whole program with link-time optimization passing the option -d:lto to the compiler, that way the C backend can automatically inline all proc code, even procs from imported modules. One more option is to use templates instead of tiny procs — templates always do a plain code substitution, so templates may behave very similar to inline procs. We will discuss templates later. The following example shows how we can apply the inline pragma to procedures and functions:

proc max(a, b: int): int {.inline.} =
  if a < b: b else: a

Note that functions from shared libraries can not be inlined, so calling external C functions, directly or indirectly, may be slower than expected.

Recursion

Procedures and functions can call themself in a repetitive manner, which is called recursion. Obviously, there must exist some condition that finally stops the recursion, without the proc would call itself, again and again, for each call some data would have to be stored on the stack, at least the proc return address, so finally the stack would overflow, and the program would crash. Generally, recursion should be used only for cases where it really helps to simplify the algorithm. In part V of the book, in the section about the various sorting algorithms, we will discover some useful applications for recursion. In most cases, a plain iterative algorithm is faster than a recursive one, because all the overhead with many proc calls is avoided for plain iterative solutions. But sometimes recursive algorithms are easier to understand, or programming an iterative solution may be really complicated.

As one of the most simple algorithms, we will present here the recursive fac() function:

proc fac(i: int): int =
  if i < 2:
    1
  else:
    i * fac(i - 1)

That function should terminate, as we only call itself again with a decreased argument. Of course, using recursion here is not really smart, it should be easy for you to convert the proc into an iterative solution without recursion. Note that recursive procs can never be inlined!

Converters

Nim’s converters are a special variant of functions, that are called automatically by the compiler when argument types do not match.

converter myIntToBool(i: int): bool =
  if i == 0:
    false
  else:
    true

proc processBool(b: bool) =
  if b:
    echo "true"
  else:
    echo "false"

var i = 7
processBool(i)
if i:
  echo "true"
else:
  echo "false"

With the above converter, we can pass an integer to a proc that expects a boolean parameter, and we can even use an integer as a logical expression in an if condition in the same way as it is done in C language. Converters do work only in a direct way, that is automatic chaining is not supported: If we have one converter from character to integer and one from int to boolean, that does not mean that we can pass a character to a proc that expects a boolean. We would have to declare one more converter that directly converts a character to a boolean.

Whenever we do consider using converters, we should think twice — converters may be confusing, may have some strange effects, and may increase compile-time.

Maybe you wondered why we wrote the above converter in such a verbose way? Well it was done intentionally, but you are right of course, we can write it just as

converter myIntToBool(i: int): bool =
  i != 0

Object-Orientated Programming and Inheritance

Object-Orientated Programming and Inheritance became very popular in the early nineties of the last century. Java is a prominent representative of that programming paradigm, but most languages created in the nineties of the last century support it, like C++, Ruby, and Python.

The idea of OOP is that objects and procedures working on that objects are grouped into classes and that classes can be extended with additional data fields and with additional procedures. In OOP, procedures and functions are often called methods, and data fields are called members. Sometimes the members are completely hidden and are accessed only by so-called getter and setter methods. That is called encapsulation. Encapsulation allows hiding implementation details, so that that details may change when necessary, without that the change of internal details becomes visible to users of that class so that the users can use the class without noticing the change. Getters and setters also help hide internal details, and they ensure that the class is always in a consistent and valid state.

An important property of OOP is dynamic dispatch: When we create various subclasses of a common parent class, and we have defined methods for all the subclasses, then we can have collections of instances of different subclasses, and the compiler can automatically ensure that always the matching method for each instance is called.

A classical example is a drawing program, where we have different geometrical shapes like rectangles, circles, and many more. All the geometrical objects are stored in some form of a list, and when we want to draw all of them on the screen, then we have to call only an unspecific draw() method, and the compiler ensures that for each shape the matching draw method is called. In Nim, that may look like

type
  Shape = ref object of RootRef

  Rectangle = ref object of Shape
    x, y, width, height: float

  Circle = ref object of Shape
    x, y, radius: float

  LineSegment = ref object of Shape
    x1, y1, x2, y2: float

method draw(s: Shape) {.base.} =
  # override this base method
  quit "to override!"

method draw(r: Rectangle) =
  echo "drawing a rectangle"

method draw(r: Circle) =
  echo "drawing a circle"

method draw(r: LineSegment) =
  echo "drawing a line segment"

proc main =
  var l: seq[Shape]
  l.add(Rectangle(x: 0, y: 0, width: 100, height: 50))
  l.add(Circle(x: 60, y: 20, radius: 50))
  l.add(LineSegment(x1: 20, y1: 20, x2: 50, y2: 50))

  for el in l:
    draw(el)

main()

The output of that program is

drawing a rectangle
drawing a circle
drawing a line segment

So we can have a sequence of the base type, add various subtypes, and then iterate over the list to draw all the various subtypes. Of course, in the same way, we could do many more tasks like moving, rotating, or storing all the objects in one call. The compiler does the right dynamic dispatching for us, we have just to provide all necessary methods. The need for the base method seems to be a bit strange, some other OOP languages do not need that. The base method is marked by a {.base.} pragma, we will discuss the purpose of pragmas later. In the example, we have used only one level of sub-classing, but of course, we can use many levels, for example, we can again subclass the Circle by a FilledCircle with a color field.

The OOP coding style can be very convenient for some tasks, one important use case could be graphical user interfaces, where the graphical elements like labels, buttons, and frames built in a natural way a hierarchical structure. Another typical use case is a drawing application, with code similar to our basic example.

Note that the OOP style only works with ref objects, but not with value objects. The obvious reason is that we can have collections of different subtypes stored in arrays or sequences only for ref objects, as in arrays and sequences all element types have to have equal size. For references, that is the case, as references are basically pointers. But different value types would have different sizes. Linked lists would be no better solution, as again we can not build lists with value objects.

For maximum performance, OOP code with ref objects is generally not optimal, as the dispatching itself needs some time, and as the ref objects are not contained in a single block of memory, but are distributed in the whole RAM, which is not cache friendly.

Other Builtin Data Types

Tuple Types

Tuples are heterogeneous container types similar to the struct type in C. As Nim’s object type creates no overhead as long as we use no inheritance, and so also directly corresponds to the C struct type, tuples are very similar to Nim’s objects. The biggest advantage of tuples is, that we can create anonymous tuples and that Nim supports the automatic unpacking of tuple variables into ordinary unstructured variables.

Compared to objects, tuples do support no inheritance at all, all the tuple fields are always visible, and different tuple types are regarded as identical when all the field names and field data types match. Remember that two different object types are always distinct in Nim, even when the actual type definition looks identical.

We can define tuple types in the same way as we define objects, or we can use the tuple[] constructor. Additionally, we can define anonymous tuples just by enclosing their field types in round brackets. The fields of tuple types can be accessed by field names as we do it for objects, or we can access the fields with constant indices starting at zero.

type
  Move = tuple # the object definition syntax
    from: int
    to: int
    check: bool

type Move = tuple[from: int, to: int, check: bool] # equivalent tuple constructor syntax

proc findBestNextMove(): (dest: int; check: bool) =
  ...

let (dst, check) = findBestNextMove()

In the code example above, we show two equivalent ways to define a tuple type, but actually, we do not use that type at all but return an anonymous tuple from our proc, that is a pair of an int and a bool.

Using automatic tuple unpacking and type inference, our dst and check variables get the data types int and bool.

Tuples are also useful when a function has to return a value and also an error state, or if it may not be able to return something at all in a special case. For reference types, we could return nil then, but for results of value type like int or float, we may not have a well-defined error indicating constant, so we can return a tuple with an additional bool indicating success or error. But of course, we could use exceptions instead, or we could use Nim’s option type instead. We will learn more about that later.

Here are two examples, which use a tuple as a proc parameter:

proc p1(x: tuple[i: int, j: int]): int =
  x.i + x.j

echo p1((7, 7))

proc p2(x: (int, int)): int =
  x[0] + x[1]

echo p2((7, 7))

Proc p1() creates a tuple type using the tuple constructor syntax with named fields, so in the proc body we can access the fields by their names, while proc p2() uses an anonymous tuple, and so has to access the fields by constant indices. Both procs are called with an anonymous tuple parameter.

Object Variants

Nim’s object variants, sometimes also called sum types or abstract data types (ADT), are an advanced and type save variant of the union type known from C. The basic idea is that we may use value types that may store similar, but not identical data. Untyped languages like Ruby or Python allow that of course, and we can do it in Nim with ref types and inheritance too, as we showed in a previous section with our Shape base type and various geometric shapes. We could store that ref types in arrays or sequences or linked lists and use dynamic dispatch for processing the various subtypes. That is convenient but gives not maximum performance due to dynamic dispatch at runtime and bad cache use. So we may like to have a value type with different content so that we can store all the value types in a seq and all entities reside in a compact block of memory for good cache use.

type
  ShapeKind = enum
    line, rect, circ

  Shape = object
    visible: bool
    case kind: ShapeKind
    of line:
      x1, y1, x2, y2: float
    of rect:
      x, y, width, height: float
    of circ:
      x0, y0, radius: float

proc draw(el: Shape) =
  if el.kind == line:
    echo "process line segment"
  elif el.kind == rect:
    echo "process rectangle"
  elif el.kind == circ:
    echo "process circle"
  else:
    echo "unknown shape"

var
  s: seq[Shape]
s.add(Shape(kind: circ, x0: 0, y0:0, radius: 100, visible: true))
for el in s:
  draw(el)

Objects variants can have common fields like the boolean state visible above, but the other fields are not allowed to use the same names, so we had to use x0 and y0 for the names of the center coordinates in the circle variant.

As you can see, we can store all the different object variants as value objects in a sequence and iterate over it. Note that object variants may waste some storage, as all variants are silently enlarged to have the exact same size, so that all variant types can be stored in arrays or sequences, and can be passed as proc parameters in the same way to the same proc.

Iterators

In the section For Loops and Iterators, we used a for loop to iterate over the individual characters of a string. For loops are useful for various iteration purposes, e.g. to iterate over container types like strings, arrays, and sequences, or over a numeric range, and other countable entities. We could do the same with a while loop, but using a for loop is often more convenient and less error-prone — we do not have to care for increasing a loop variable and for the stop condition.

Nim’s for loops are built on iterators, that is, whenever a for loop is executed, an iterator is used under the hood. Some iterators are used explicitly in for loops, e.g. countup() of Nim’s standard library, others like items() or pairs() are executed implicitly when no explicit iterator name is specified.

The creation and use of iterators is very easy in Nim. Before we will discuss all the details and some restrictions of iterators, and the important differences between inline and closure iterators, we will give a small example:

We have already used some of Nim’s standard iterators to iterate over the characters of a string or the content of a sequence.

In an earlier section of the book, we declared a procedure that extracts all the decimal digits from a string. We can do the same with an iterator:

iterator decDigits(s: string): char =
  var pos = 0
  while pos < s.len:
    if s[pos] in {'0' .. '9'}:
      yield(s[pos])
    inc(pos)

for d in decDigits("df4j6dr78sd31tz"):
  stdout.write(d)
stdout.write('\n')

The definition of an iterator is very similar to the definition of a proc or function, but while a function returns a result only once to the caller, an iterator uses the yield statement to give data back to the call site multiple times, instead of returning just once.

Whenever a yield statement is reached in the body of the iterator, the yielded data is bound to the for loop variable(s), the body of the for loop is executed, and at the end of the for loop body control returns to the iterator, that is execution continues directly after the yield statement. The iterator’s local variables and execution state are automatically saved between calls. The iteration process continues until the end of the body of the iterator declaration is reached and the iterator terminates.

Iterators are used in for loops to iterate over containers, ranges or other data. After the for keyword, we specify one or more arbitrary variable names, which we then can use in the body of the for loop to access the yielded value(s). The data type of this iteration variable(s) is inferred from the iterators return type, and its scope is limited to the body of the for loop.

Nim’s standard library defines for container types like strings, arrays and sequences iterators named items() and pairs() — items() is the default name when a for loop with only one variable is used, and pairs() is the default name when two variables are used, e.g. the index position and the character when iterating over a string.

In Nim’s standard library, you may find items() and pairs() iterators like these two:

iterator items(a: string): char =
  var i = 0
  while i < len(a):
    yield a[i]
    inc(i)

iterator pairs(a: string): tuple[key: int, val: char] =
  var i = 0
  while i < len(a):
    yield (i, a[i])
    inc(i)

var s = "Nim is nice."
for c in items(s):
  stdout.write(c, '*')
echo ""
for i, c in pairs(s):
  echo i, ": ", c

In the example above, we specified the iterator names items() and pairs() explicitly in the for statement, but as these names are the defaults, we can just write for c in s: and for i, c in s:.

The two iterators in the example code from above use as argument a value type and return the single characters as a value type. This way, we can not modify the string content. When we intend to modify the content of a container by use of an iterator, we have to pass the container as a var parameter and return the elements as var also. By convention for iterating over mutable containers the iterator names mitems() and mpairs() are used, where the leading m stands for mutable. We have to specify these names explicitly:

iterator mitems(a: var string): var char =
  var i = 0
  while i < len(a):
    yield a[i]
    inc(i)

iterator mpairs(a: var string): tuple[key: int, val: var char] =
  var i = 0
  while i < len(a):
    yield (i, a[i])
    inc(i)

from strutils import toLowerAscii
var s = "NIM"
for i, c in mpairs(s):
  if i > 0:
    c = toLowerAscii(c)
echo s # Nim

Whenever we iterate over a container, we should not delete, insert or append elements to the container, as that may confuse the loop inside the iterator body. Iterators of Nim’s standard library check the length of the container and generate an exception when the length changes during the iteration.

Nim differentiates between inline and closure iterators. When a for loop uses an inline iterator, then the actual iterator loop is inlined in the for loop body in a way that for each yield statement in the iterator body, the body of the for loop is executed. Actually the for c in items(s): stdout.write(c, '*') in our example from above is rewritten by the compiler into a code block like

var i = 0
while i < len(a):
  var c = a[i]
  echo c, '*'
  inc(i)

That is, the body of the for loop is inlined into the loop of the iterator.

This results in very fast code with no overhead, but similar to the use of templates, this increases the total code size. Actually, when the iterator should use multiple yield statements, then the code of the body of the for loop is inserted for each yield statement.

Inline iterators are currently the default iterator type, so the iterators of the examples above are all inline iterators.

Closure iterators behave more like procedures, the iterator is actually invoked, which costs some performance. We can use all the iterators of the examples from above as closure iterators by applying the closure pragma as in iterator items(a: string): char {.closure.} =.

Closure iterators behave like objects, we can assign instances of closure iterators to variables, and then call the instances explicitly:

iterator myCounter(a, b: int): int {.closure.} =
  var i = a
  while i < b:
    yield i
    inc(i)

for x in myCounter(3, 5): # ordinary use of the operator
  echo x

echo "---"
var counter = myCounter # use of an iterator instance
while true:
  echo counter(5, 7)
  if counter.finished:
    break

which gives us this output:

3
4
---
5
6
0

Here, we have used the finished() function to check if the iterator is done.

Actually, finished() returns true only, when the iterator has already failed to yield a valid value, and not already when the last valid value was yielded. That is, why we get in the example above as the last value the invalid value zero.

We can avoid this behavior, when we rewrite the loop as

var counter2 = myCounter
while true:
  let v = counter2(5, 7)
  if counter2.finished:
    break # v is invalid
  echo v

Closure iterators are resumable functions, and so one has to provide the arguments to every call. To get around this limitation, one can capture parameters of an outer factory proc:[38]

proc mycount(a, b: int): iterator (): int =
  result = iterator (): int =
    var i = a
    while i < b:
      yield i
      inc(i)

var c1 = mycount(5, 7)
for i in c1():
  echo i

echo "---"

var c2 = mycount(2, 5)
while true:
  let v = c2()
  if c2.finished:
    break # v is invalid
  echo v

In this example from the compiler manual, the proc mycount() captures the bound for the counter. When we compile and run the code above, we get:

5
6
---
2
3
4

At the end of this section we will list some properties of iterators: Iterators have their own namespace, so we can freely use for procs and iterators the same names. Iterators have no predefined result variable and do not support recursion. Inline iterators can be used only inside for loops, and can not be forward declared, because the compiler must be able to inline an iterator. (This restriction will be gone in a future version of the compiler.) And iterators do not support recursion. Closure iterators are not supported by the JS backend, and cannot be executed at compile time. Inline iterators are second-class citizens; They can be passed as parameters only to other inlining code facilities like templates, macros, and other inline iterators. In contrast to that, a closure iterator can be passed around more freely.

Templates

Nim templates are very different from C++ templates! In C++ templates are used for generic programming — a style of computer programming in which algorithms are written in terms of types to-be-specified-later that are then instantiated when needed for specific types provided as parameters.[39] This is just called generics in Nim and other programming languages, we learned about Nim’s generics earlier in this book.

Nim templates are a simple, parameterized code substitution mechanism, and are used similarly as procedures. The syntax to invoke a template is the same as calling a procedure. But while procedures built a single block of code that is then called multiple times, templates work more like C macros as a (textual) code substitution. Wherever we invoke a template, the template source code is inserted at the call site. In this way, Nim templates have indeed some similarity to C macros. But while C macros are executed by the C pre-processor and can do only plain source text substitutions, Nim templates operate on Nim’s abstract syntax trees, are processed in the semantic pass of the compiler, integrate well with the rest of the language, and share none of C’s preprocessor macros flaws.

In some way, Nim templates are a simplified application of Nim’s powerful macro and meta-programming system, which we will discuss in detail in part VI of the book.

In C we could use the "#define" preprocessor directive to define two simple C macros.

#define PI 3.1416
#define SQR(x) (x)*(x)

The C pre-processor would then replace the symbol PI in the C source code with the float literal 3.1416 before the code is processed by the C compiler. And as the C pre-processor can recognize some simple form of parameters, it would replace SQR(a + b) with (a+b)*(a+b).

In Nim we would define a const for PI and use a generic proc or a template for SQR():

const PI = 3.1416
proc sqr1[T](x: T): T = x * x
template sqr2(x: typed): typed = x * x

Here the sqr2() template uses the special typed parameter, which specifies that the parameter has a well-defined type in the template body, but that arbitrary data types are accepted. So sqr1() and sqr2() would work for all numeric types and also for other data types for which we have defined a * operation. When there is no * operator defined for the passed data type, the compiler will give an error message.

Nim templates accept like procs all of Nim’s ordinary data types, and additional the abstract meta-types typed and untyped. The abstract data types typed and untyped can be used only for the types of template and macro parameters, but not for parameters of procedures, functions, iterators, or to define variables.

We will explain the differences between typed and untyped in detail later in this section — the short version of the explanation is, that typed template parameters must have a well-defined data type when we pass it to the template, while untyped parameters can be passed as still undefined symbolic name also.

So we can in principle replace each procedure or function definition with a template. The important difference between procedures and templates is, that ordinary procs are instantiated only once, generic procs may be instantiated for each data type with which they are used, but templates are instantiated for each invocation of the template. The compiler creates for each defined proc some machine code, which is executed whenever the proc is called. But for templates, the compiler does some code substitution — the source code of the template is inserted where the template is invoked. This avoids the need for an actual jump to a different machine code block when a procedure is called but increases the total code size for each use of a template. So we would typically avoid templates that contain a lot of code and are used often.

For each ordinary proc, one block of machine code instructions is generated, and when the proc is called, program execution has to jump to this block, and back when the proc execution is done. This jumping involves some minimal overhead, which is noticeable for tiny procs called frequently. To avoid this overhead, we may use a template instead, or we may use inlined procs, which we discussed in the previous section. Proc inlining can be done by the compiler automatically when the proc is defined in the source code file where it is used, or when we mark the proc with the inline pragma. Additionally, when we compile our program with -d:lto, the compiler can inline all procedures and functions. Generally, the compiler should know well when inlining makes sense, so in most cases, it makes not much sense to just use templates instead of (small) procs just to avoid the proc call overhead.

Templates can be used as some form of an alias. Sometimes we have nested data structures, and would like to have a shorter alias for the access of fields:

type
  Point = object
    x, y: int

  Circle = object
    center: Point

template x(c: Circle): int = c.center.x

template `x=`(c: var Circle; v: int) = c.center.x = v

var a, b: Circle

a.center.x = 7
echo a.center.x

b.x = 7
echo b.x

The two templates simplify the access of field x, and as templates are pure code substitution, their use costs no performance. Since version 1.6 Nim has also the with macro, which can be also used to save some typing. Note that in the second template, we have called the second int parameter v — calling them x would give some trouble:

Error: in expression 'b.center.7': identifier expected, but found '7'

Nim’s system module uses templates to define some operators like

template `!=` (a, b: untyped): untyped =
  not (a == b)

This way != is always the opposite of ==, so when we define the == operator for our own custom data types, != is available for free.

In some situations, using templates instead of procs can avoid some overhead. Let us investigate a log() template, that can print messages to stdout, when a global boolean constant is set to true:

const
  debug = true

template log(msg: string) =
  if debug: stdout.writeLine(msg)

var
  x = 4
log("x has the value: " & $x)

Here log() is called with the constructed argument ("x has the value: " & $x), which implies a string concatenation operation at runtime. As we use a template, the invocation of log("x has the value: " & $x) is actually replaced by the compiler with code like

  if debug: stdout.writeLine("x has the value: " & $x)

So when debug is set to false, absolutely no code is generated. For an ordinary, not inlined proc, the situation is different, the expensive string concatenation operation would always have to be performed, but the log() proc would immediately return for debug == false. What exactly would happen when log() is an inlined proc may depend on the actually used compiler backend.

Note that the delayed (lazy) parameter evaluation for template parameters can have disadvantages: When we modify the log() template like

template log(msg: string) =
  for i in 0 .. 2:
    stdout.writeLine(msg)

var x = 4
log("x has the value: " & $x)

the expensive string concatenation operation would be done in principle three times in the template body.[40] While for a proc the already evaluated parameter would be passed. So when we access a parameter multiple times inside a template, it can make sense to assign the parameter to a local variable and then use that variable only.

Templates can inject entities defined in the template body into the surrounding scope. By default, variables defined in the template body are not injected in the surrounding scope, but procs are:

template gen =
  var a: int
  proc maxx(a, b: int): int =
    if a > b: a else: b

gen()
echo maxx(2, 3)
# echo a

The call echo maxx(2, 3) compiles and works, while echo a complains about an undefined symbol.

A very special property of templates and macros is, that we can pass to them code blocks when we use untyped for the type of the last parameter:

template withFile(f: untyped; filename: string; actions: untyped) =
  var f: File
  if open(f, filename, fmWrite):
      actions
      close(f)

withFile(myTextFile, "thisIsReallyNotAnExistingFileWithImportantContent.txt"):
  myTextFile.writeLine("line 1")
  myTextFile.writeLine("line 2")

The template withFile() from the above example has three parameters — a parameter f of untyped type, a filename of string type, and as the last parameter one more untyped parameter, which we called actions. For this last untyped actions parameter, we can pass an indented code block.

When we invoke the withFile() template, we pass the first two parameters in the well-known way by putting them in a parameter list enclosed in round brackets. But instead of passing this way also the final actions parameter, we put a colon after the parameter list and pass this way the following indented code block as the last untyped parameter. In the body of the above template, we have an open() call which opens a file with the specified filename and the fmWrite mode, then executes the passed code block, and finally closes the file. The first parameter of our withFile() template has also a special property: As we use untyped for the f parameter, we can pass the still undefined symbol myTextFile to the template. In the template body, this symbol is used as a variable name, and our two writeLine() proc calls can use it to refer to the file variable.

As Nim templates are hygienic, the instance of the file variable created in the body of our template can be used by the passed code block, but it actually exists only in the template and pollutes not the global namespace of our program.

We can use templates to create new procs. An example is lifting procs like math.sqrt() that accepts a scalar parameter and returns a scalar value, to work with arrays and sequences. The following example is taken from the official tut2 tutorial:

from std/math import sqrt

template liftScalarProc(fname) =
  proc fname[T](x: openarray[T]): auto =
    var temp: T
    type outType = typeof(fname(temp))
    result = newSeq[outType](x.len)
    for i in 0 .. x.high:
      result[i] = fname(x[i])

liftScalarProc(sqrt)   # make sqrt() work for sequences
echo sqrt(@[4.0, 16.0, 25.0, 36.0])   # => @[2.0, 4.0, 5.0, 6.0]

The template called liftScalarProc() creates a generic proc, that accept as parameter an openArray[T] and returns a seq[T]. Well, we should be able to understand the basic ideas used in that code, but it is still fascinating that it really works.

Typed vs untyped parameters

Parameters passed to templates can have all the data types that we can use for procs including special types like openarray and varargs, and we can use as types additional the symbols untyped, typed or typedesc.

The typedesc type can be used to pass type information to the template, e.g. when we want to create a variable of a special data type. The "meta-types" typed and untyped are used, when we want to create some form of a generic template, that can accept different data types. Actually, the distinction between typed and untyped parameters is not that difficult and important for templates as it is for macros, in most cases, it is just clear if we need the typed or untyped parameter type for a template, or both work fine. We discuss the differences between typed and untyped in much more detail in part VI of the book when we discuss macros and meta-programming.

The following example demonstrates the use of the untyped and the typedesc parameter:

template declareInt(n: untyped) =
  var n: int

declareInt(i)
i = 3
echo i

template declareVar(n: untyped; t: typedesc) =
  var n: t

declareVar(x, float)
x = 3.0
echo x

As the parameter n is untyped, the compiler allows us to pass an undefined symbol to the template. If we changed the parameter type to typed, the compiler would complain with a message like Error: undeclared identifier: 'i'

For the second template, called declareVar(), we use an additional parameter of typedesc type, so that the template can create for us a variable of just the passed data type.

Citing the manual: "An untyped parameter means that symbol lookups and type resolution is not performed before the expression is passed to the template. This means that undeclared identifiers, for example, can be passed to the template. A template where every parameter is untyped is called an immediate template. For historical reasons, templates can be explicitly annotated with an immediate pragma and then these templates do not take part in overloading resolution and the parameters' types are ignored by the compiler. Explicit immediate templates are now deprecated. For historical reasons, stmt was an alias for typed and expr was an alias for untyped, but they are removed."

Passing a code block to a template

In the withFile() example above, we showed that we can pass a block of statements as the last argument to a template following the special : syntax. To demonstrate the difference between code blocks of typed and untyped data type we will cite the compiler manual, see https://nim-lang.org/docs/manual.html#templates-passing-a-code-block-to-a-template:

Usually, to pass a block of code to a template, the parameter that accepts the block needs to be of type untyped. Because symbol lookups are then delayed until template instantiation time:

template t(body: typed) =
  proc p = echo "hey"
  block:
    body

t:
  p()  # fails with 'undeclared identifier: p'

The above code fails with the error message that p is not declared. The reason for this is, that the p() body is type-checked before getting passed to the body parameter, and type checking in Nim implies symbol lookups. The same code works with untyped as the passed body is not required to be type-checked:

template t(body: untyped) =
  proc p = echo "hey"
  block:
    body

t:
  p()  # compiles

Passing operators to templates

One more use case for templates with untyped parameters is the generation of math operations for custom data types. Let us assume that we have created a custom Vector object, for which we have to define addition and subtraction operations. Instead of writing code for both cases, we can use a template and pass the actual math operator as untyped parameter:

type
  Vector = object
    x, y, z: int

template genOp(op: untyped) =
  proc `op`(a, b: Vector): Vector =
    Vector(x: `op`(a.x, b.x), y: `op`(a.y, b.y), z: `op`(a.z, b.z))

genOp(`+`)
genOp(`-`)

echo `+`(2, 3) # 5

var p = Vector(x: 1, y: 1, z: 1)
var p2 = p + p
echo p2 # (x: 2, y: 2, z: 2)

This works, because for operators math like 1+2 can be written as +(1, 2) and because we can pass such an operator as untyped parameter to a template.

Advanced template use

For the advanced template stuff, you should consult the compiler manual.

This includes the symbol binding rules, identifier construction in templates, lookup rules for template parameters, hygiene in templates, use of the inject pragma, and limitations of the method call syntax.

All this is explained well in the compiler manual, so it makes no sense to repeat it here. And it makes more sense to read it when you actually have problems with the (default) behavior of templates in special situations.

Casts and Type Conversion

While we have various types of casts in C++, we have only one cast, and type conversions in Nim. In Nim, the cast just reinterprets the same bit pattern for another data type. For example, the boolean value false is internally encoded as a byte with all bits cleared, and true is encoded as a byte with all bits but the least significant one cleared. We could cast a bool to an int8 of the same size and would get a number with a decimal value of 0 or 1. Casting is not a real operation at all, as nothing is really done. We watch the same bit pattern, just from a different perspective. But casting is dangerous, it violates the safe type system of the language, and it can go very wrong: Can we cast between float64 and int64? Well, they have the same size, and both are numbers. We can cast, but the result would be far away from what we may expect. While int64 has the well-known and simple value encoding, that is rightmost bit stands for 2^0, the next bit for 2^1 and so far, the encoding of floating-point numbers is much more difficult and has not such a simple scheme. In floats, some bits represent the so-called mantissa, and some bits represent the exponent. When we cast, we may again get a number, but the value is not easily predictable. We have to be very careful when we cast between types of different sizes. Nim may permit that, but we have to think about what may really happen. When we cast between a bool and an int64, in one direction 7 bytes have to be ignored, and in the other direction for 7 missing bytes there is some padding necessary. We do a cast by writing after the keyword cast in square brackets the desired type, followed by parenthesis enclosing the source variable:

var i: uint8 = cast[unint8](myBoolVar)

Totally different from casting is type conversion. We can convert integers to floating-point numbers without problems, for the conversion we use the type like a proc call, that is int(myfloat) or float(myInt) — of course, we could use method call syntax like myInt.float instead. Type conversion is some effort for the CPU, but most advanced CPUs should have fast instructions for basic conversion.

Nim generally only allows type conversions that involve not too much effort. So we should not expect var i: string ="1234"; echo i.int * 7 to be available. Such a conversion is expensive, at runtime it costs many CPU cycles, as we would have to extract the digits, multiply with their weight and sum them up. So for that operation, functions like parseInt() are available from the Nim standard library, which accepts a string as an argument and returns an int. There exist different variants of parseInt(), one may raise exceptions for invalid input, and the other may return a boolean.

Bitwise Operations

All systems programming languages, and most other programming languages, have support for bit manipulation operations, which includes querying and setting individual bits of variables, and combining the bits of two or more variables. As the CPU hardware supports these operations directly, these operations are very efficient. In the C programming language, operators like &, |, <<, >>, ^, ~ are used for bit-wise and and or operations, for shifting all the bits of a variable to the left or to the right, and for the process of inverting all the bits and for applying the exclusive-or operation on the bits of two operands. Actually, for the right shift operation, we have to distinguish between a logical and an arithmetic shift: For a logical shift the bit pattern is only moved right, and the leftmost bit is always cleared. But for an arithmetic shift, the leftmost bit may stay set when it was set before, indicating a negative number in the case of a numeric variable. In C the actual behavior for a >> shift right operation may be implementation-dependent.

Nim prefers to use textual operators instead of cryptic symbols, so the logical operators and, or and not have overloads to work on the actual bit pattern of integer variables instead of on boolean values, and for logical left and right shifts the operators are called shr and shl. For shl shifted in bits from the right are always cleared, while shr shifts in cleared bits from the left for unsigned arguments, but preserves the leftmost set bit for signed arguments, which corresponds to an arithmetic shift operation. The Nim standard library also provides an ashr() function for arithmetic shifts, but that one seems to be a legacy.

from strutils import toBin
var i = 1.int8 # 0b00000001
i = i shl 7 # 0b10000000
i = i shr 2 # 0b11100000 as sign is preserved
echo i.toBin(8)
var j: uint8 = 0b11111111
j = j shr 2 # 0b00111111, div 4 for unsigned int
echo j.int8.toBin(8)

The bit-wise operators and, or, and not behave very similarly to the boolean ones, but the operation is performed for all the bit values instead of two boolean operands. The shift operators require a right-hand operand specifying how many positions the bit pattern of the integer variable on the left should be moved. As the shr operator preserves the leftmost sign bit for each individual shift when applied to a signed integer argument, we get a value with the three leftmost bits set in the above example. For showing the bit pattern, we used the toBin() function in the above code, the second parameter determines how many bits are actually printed. Remember, that for unsigned numbers, shl by one position is a multiplication by two, and shr by one position is a division by two. Negative numbers are not allowed for the number of bits to shift — i = i shl -1 does compile, but the result is always zero. For all the shift operations, n shifts each by one position would give the same result as one single shift by n positions. For most modern CPU hardware, all the bit shifting operations are very fast and generally take only one clock cycle, independent of how many positions we move the bit pattern and independent of it is a logical or an arithmetic shift operation.

We can use the and and the or operator to extract single bits, or to set single bits:

var a = 3 # two rightmost bits, at position 0 and 1 are set
var b = a and 2 # extract bit 1, so b gets the value 2
b = a and 4 # extract bit 3, which is unset, so result is 0
b = a or (4 + 8) # result is \b00001111, decimal 15

This should be enough to teach you the most basic bit operations. Actually, we need these operations not that often, but we should be aware of their existence. The overloading of the and, the or and the not operator for signed and unsigned integer numbers may be convenient, but it may sometimes lead to confusion when we intend to do boolean operations, but instead actually do operations on bit patterns. It was suggested to call the operators bitand, bitor and bitand instead, and indeed the bitops module of Nim’s standard library defines operators with these names and provides additional, more useful bit operations, including counting the number of set bits in a variable or determining the number of leading zero bits. These operations are not needed that frequently, but sometimes they can be very useful, and they are supported by fast CPU instructions on modern PC hardware. Note that while we have shown these bit operations on integer numbers only, you can always cast other data types to integers and then apply these operations as well.

Exceptions

When we execute our program code, sometimes something can go wrong: We may be unable to open a file, have an unexpected division by zero or an overflow, or get some invalid user input. There exist various strategies to handle such situations. One is to terminate our program, we may do that by a plain assert() or quit() statement. If we have absolutely no idea how to recover from an error, then that may be our best option. The user may restart the program, or the program may be restarted by some sort of supervisor program. For more expectable errors, some form of error indicator may be a better solution, for example, a parseInt() procedure may return a boolean value for success. As we have to return the numerical result for success as well, the parseInt() proc may return a tuple or may use a var parameter in which the actual integer value is returned. Whenever a proc returns a reference, then the return value nil may be used to indicate some form of error, or we may use Nim’s Option type to allow the caller to detect if a returned value is invalid.

Another popular strategy to handle error states is the use of Exceptions. When somewhere in the code path an invalid operation is detected, then that code can raise an Exception, to indicate that a serious error has occurred. This raised error may be handled elsewhere in the program, or if it is not handled at all, the raised Exception will finally cause a program termination.

Let us start again with a small example:

proc charToInt(c: char): int =
  if c in {'0' .. '9'}:
    return ord(c) - ord('0')
  raise newException(OSError, "parse error")

proc main =
  while true:
    stdout.write("Please enter a single decimal digit: ")
    let s = stdin.readline
    try:
      echo "Fine, the number is: ", charToInt(s[0])
    except:
      if s.len == 0:
        break
      echo "Try again"

main()

The charToInt() proc raises an Exception when the passed character is not a decimal digit. As the main program knows that charToInt() may raise an Exception, it encloses the charToInt() call in a try/except block: If code in the try block raises an Exception, then the program execution proceeds in the except: block.

The use of Exceptions seems to be a good idea to handle some kind of rare errors, and most modern programming languages support some form of raising and catching Exceptions. But there have also been some critics: This form of raising Exceptions, and catching them with try/except blocks, breaks the regular control flow of the program, which may make it hard to reason about all the possible code paths. For this reason, the popular Go programming language was initially released with Exception handling explicitly omitted, with the developers arguing that it obfuscated control flow. Actually, the Nim compiler can help us with the bookkeeping of all the involved Exceptions by its effect system, which is described in much detail in the compiler manual and which we will discuss briefly in the next section. Nim’s Exception tracking is part of Nim’s effect system — we can annotate each proc with all the various types of Exceptions that it may raise, and the compiler can help us with this annotation and verify that it is correct.

Defects and catchable Errors

Nim’s strategy for the handling of Exceptions has changed a bit in the last few years. Nim differentiates now between catchable errors, and defects, which may not be catchable, and are considered to be programming bugs. The prototype of a defect is the DivByZeroDefect. If we do an integer division by zero, then the most common CPUs will generate a signal and the OS will abort our program with SIGFPE. So to prevent the program abort by a possible DivByZeroDefect, we have always to ensure that for an integer division the denominator is not zero, or we let the Nim compiler do this check by compiling with option --checks:on, which cost performance and increases code size, as then for each division a check instruction is added.

In Nim, all Exceptions types are objects that inherit from the Exception type of the system module and have public name and msg fields of string type.

Exceptions that indicate programming bugs inherit from system.Defect and can be uncatchable as they can be mapped to operations that terminate the whole process, like a quit / trap / exit operation. Exceptions that indicate other, catchable runtime errors inherit from system.CatchableError.

These types are further subclassed in Defects like OverflowDefect or OutOfMemDefect, and Errors like ValueError or IOError.

Raise statement

Raising an Exception is done with the raise statement. Raise expects a heap allocated reference to an Exception object, as the lifetime of the Exception instance is unknown. Generally, we use the newException() template to create the Exception instance and set the msg field like

raise newException(IOError, "IO failed")

In principle, we could also create the Exception instance like

var
  e: ref OSError
new(e)
e.msg = "the request to the OS failed"
raise e

If no Exception name is given for a raise call, the current Exception is re-raised. The ReraiseDefect Exception is raised if there is no Exception to re-raise. It follows that the raise statement always raises an Exception. Reraising an Exception can be useful in an except block (see below), when the actual Exception type can not be handled.

Custom exceptions

Instead of raising one of the predefined Exceptions from the system module, we can create also our own variants and then raise them:

type
  LoadError = object of Exception

Try statement

In the Nim compiler manual, we have an example like this one:

var
  f: File
if f.open("numbers.txt"):
  try:
    let a = f.readLine
    let b = f.readLine
    echo "sum: ", parseInt(a) + parseInt(b)
  except OverflowDefect:
    echo "overflow!"
  except ValueError, IOError:
    echo "value or IO error!"
  except:
    echo "Unknown exception!"
  finally:
    close(f)

The code tries to read two strings from a text file that is assumed to contain numeric data and to add them after conversion to integer numbers. Three errors may occur: The reading of the strings may fail, the conversion to integers may fail, and finally, the addition may cause an overflow. To catch the possible errors, we use the try/except/finally construct. The keywords try, except, and finally are followed by a colon, and each keyword marks the start of a corresponding block of instructions — after the except keyword we can list the Error and Defect types for which the following code block should be executed.

The statements of the try block are executed in sequential order as long as no Exception is raised. If an Exception is raised and the Exception type matched any listed in an except clause, the corresponding statements are executed. If no Exception types match and an except clause with no listed Exception types is specified, the following code block is executed. The statements following the except clauses are called Exception handlers. If there is a finally clause, it is always executed after the Exception handlers.

An Exception is "consumed" in an Exception handler. However, an Exception handler may raise another Exception or re-raise the current one, which then may be caught elsewhere or generate a program termination if it remains uncaught. If an Exception is not handled, it is propagated through the call stack. This means that often the rest of the procedure - that is not within a finally clause - is not executed (if an Exception occurs).

Try expressions

Similar to we can use the if keyword as an expression, we can do the same with the try keyword. The data types of the try and the except branches have to be compatible in this case, and an optionally finally branch has to return nothing (void):

from std/strutils import parseInt
let x = try: parseInt("3.14")
        except: NaN
        finally: echo "well we tried."

let i = (try: parseInt("133a") except: -1)

Except clauses

In an except block, we can use the functions getCurrentException() to get the raised Exception, or just getCurrentExceptionMsg() to get only the error message. Or we can access the current Exception in an except block by use of the as keyword like

try:
  # ...
except IOError as e:
  # Now use "e"
  echo "I/O error: ", e.msg, " (", e.name, ')'

Imported exceptions

It is possible to raise and catch imported C++ exceptions. The Nim compiler manual has a detailed example for that, see https://nim-lang.org/docs/manual.html#exception-handling-imported-exceptions

Defer statement

The defer statement can be used to ensure that special actions like closing a file or freeing resources are always executed. The defer statement is rewritten by the compiler into a try/finally construct.

proc main =
  var f = open("numbers.txt", fmWrite)
  defer: close(f)
  f.write "abc"
  f.write "def"

Is rewritten to:

proc main =
  var f = open("numbers.txt", fmWrite)
  try:
    f.write "abc"
    f.write "def"
  finally:
    close(f)

Using defer is shorter, but with try/finally it is more obvious what is going on, so some people recommend ignoring the defer statement. Actually, task like closing files should soon be performed by Nim’s destructors automatically, so defer may get deprecated.

References:

Effect system

Destructors

Destructors and finalizers are used for automatic resource management. For example, files can be closed automatically when a file variable goes out of scope, or when we create high-level Nim bindings to C libraries we can use finalizers or destructors to deallocate entities of the C libs when a corresponding Nim (proxy) object is freed. Libraries like the gintro GTK bindings make use of this.

Finalizers are procedures that can be passed as a second, optional parameter to the system new() proc. That way, the finalizer proc is attached to the data type of the variable which we pass as the first parameter to new() and that finalizer proc is automatically called whenever that variable is freed by the Nim memory management system. As finalizers are passed as a parameter to a new() call, and new() is only used for references, finalizers work only for ref data types.

Destructors do not have this restriction. We define the destructor for a value type, but it is also called for reference types by the compiler.

Starting with version 1.4, Nim got scope-based resource management, when the program is compiled with --gc:arc or --gc:orc. In that case, variables are immediately deallocated when they go out of scope, and if a destructor was defined for the data type of that variable, it is called automatically.

For the programming language C++, it is a common practice that resources like files are closed and released automatically by destructors when they go out of scope, and now this is also possible for Nim. To make use of destructors for our own data types, we have to define a proc called =destroy which gets an instance of our data type passed as a var value object:

type
  O = object
    i: int

proc `=destroy`(o: var O) =
  echo "destroying O"

import random

proc test =
  for i in 0 .. 5:
    if rand(9) > 1:
      var o: O
      o.i = rand(100)
      echo o.i * o.i

randomize()
test()

In the for loop, we enter a new scope when the if condition is evaluated to true, and at the end of the if block, we leaf the scope and the destructor is called automatically. Inside the destructor proc, we could do some cleanup tasks, close files, and release resources. Destructors are also called when ref objects go out of scope:

type
  O = ref object of RootRef
    i: int

proc `=destroy`(o: var typeof(O()[])) =
  echo "destroying O"

import random

proc test =
  for i in 0 .. 5:
    if rand(9) > 1:
      var o: O = O() # new O
      o.i = rand(100)
      echo o.i * o.i

randomize()
test()

To use destructors, we have to compile our program with the option --gc:arc or --gc:orc, otherwise the specified destructor procs are just ignored. In our code, we can test for working destructors with a construct like when defined(gcDestructors):.

Note that destructors do not work for plain pointer types:

type
  O = object
    i: int
  OP = ptr O

proc `=destroy`(o: var O) =
  echo "destroying O"

import random

proc test =
  for i in 0 .. 5:
    if rand(9) > 1:
      var o: OP = create(O) # new O
      o.i = rand(100)
      echo o.i * o.i

randomize()
test()

So using destructors to release data from C libraries directly is not possible. But at least for Nim >= v1.6 destructors work for distinct pointer types:

type
  O = object
    i: int
  OP1 = ptr O
  OP = distinct ptr O

proc `=destroy`(o: var OP) =
  echo "destroying OP"

import random

proc test =
  for i in 0 .. 5:
    if rand(9) > 1:
      var o: OP = OP(create(O)) # new O
      OP1(o).i = rand(100)
      echo OP1(o).i * OP1(o).i

randomize()
test()
81
destroying OP
3600
destroying OP
2401
destroying OP
9025
destroying OP

So using destructors to destroy data from C libraries should be possible now.

Destructors and Inheritance

When we use an object-oriented programming style with subclassing of ref objects, then it is useful to know that for subclassed ref objects the destructor of the parent class is automatically invoked when we do not define our own one for our subclassed type. This works also when we import the parent type from another module, at least since Nim v1.6:

# module tt.nim
type
  O1* = ref object of Rootref
    i*: int

when defined(gcDestructors): # check not really needed, as =destroy call is just ignored when condition is false
  proc `=destroy`*(o1: var typeof(O1()[])) =
    echo "destroy O1 ", typeof(o1)
# module t.nim
import tt

type
  O2 = ref object of tt.O1
    j: int

type
  O3 = ref object
    o1: tt.O1

type
  O4 = object
    o1: tt.O1

type
  O5 = ref object of tt.O1
    x: float

when defined(gcDestructors):
  proc `=destroy`(o5: var typeof(O5()[])) =
    echo "destroy O5 ", typeof(o5)
    tt.`=destroy`(o5)

proc main =
  var o1: tt.O1
  new o1
  echo o1.i

  var o2: O2
  new o2
  echo o2.j

  var o3: O3
  new o3
  new o3.o1

  var o4: O4
  new o4.o1

  var o5: O5 = O5(x: 3.1415)
  echo o5.x

main()

When we compile module t.nim with --gc:arc or --gc:orc and run it, we get this output:

0
0
3.1415
destroy O5 O5:ObjectType
destroy O1 O1:ObjectType
destroy O1 O1:ObjectType
destroy O1 O1:ObjectType
destroy O1 O1:ObjectType
destroy O1 O1:ObjectType

So when our variables o1 to o5 go out of scope, then the destructors are called. Module tt.nim defines a ref object type, but the destructor proc takes a var value parameter. The destructor is called when a value object or a ref object goes out of scope. Our variable o1 has type tt.O1, so it was indeed expected that its destructor from module tt.nim is called. Variable o2 is a ref object with parent O1, as we define no destructor for this type, the destructor of the parent type is called. The variables o3 and o4 are of ref object and of value object types, each with a field of type O1, and for that field, the destructor for O1 is called. Finally, for type O5 we define our own destructor, which then additional calls the destructor of module tt.

Destructors are mostly used for library implementations, e.g. for a File data type, which is automatically closed when a file variable goes out of scope. As you may never have to use destructors yourself, it is not necessary to remember all these details. But it is good to know that destructors behave in a way as we may have expected, and when you later want to use a destructor in your own code, you can consult again this section or maybe better the Nim manual.

References:

Finalizers

In Nim, finalizers are procedures, that we can specify as an optional second parameter when we call the system new() proc to allocate heap memory for a reference type variable. That specified finalizer proc is then later called by the Nim memory management system when the ref variable is freed:

type
  O = ref object of RootRef
    i: int

proc finO(o: O) =
  echo "finalize O"

proc newO: O =
  new(result, finO)

proc main =
  var o = newO()
  var o2 = new(O)
  var o3 = O(i: 7)

main()
finalize O
finalize O
finalize O

The output of the above program is really surprising at first: Only for variable o we do call the proc newO() to initialize it, which then calls new() by passing a finalizer proc named finO(). For o2 and o3, we allocate memory as usual, without the use of a finalizer proc. But when o2 and o3 go out of scope, even for these two variables, the finalizer proc finO() is called. The reason for this is, that the system proc new() binds the optional finalizer proc to the data type of the passed ref variable. This binding process occurs for the first call with a passed finalizer proc, and can not be reverted. We can later call new() without a finalizer or use the similar O() call to initialize the ref variable, but that can not undo the binding. And using a different finalizer proc for the same data type would not work anymore. Passing the same finalizer proc multiple times is OK and may be a common use case, but it has no real effect, as the first call did the binding already.

This behavior of finalizers in Nim is indeed a bit confusing and error-prone. Maybe somewhere in a large program we pass a finalizer proc to new() and forget about it. Later, we use new () without a finalizer or use the O() notation to reserve the memory for our ref variable. So we think that no finalizer is involved, but as a finalizer was used somewhere at least once, it is now bound to all of our allocations of that data type. That can easily lead to bugs as the unintended called finalizers may do things that they should not do with our data.

Finalizer procs have to be defined always in the same module as the type for which the finalizer shall be used is defined:

# module tt.nim
type
  O* = ref object of RootRef
    i: int

proc fin*[T](o: T) =
  echo "finalize T"

proc newO*: O =
  new(result, fin)
import tt

type
  OO = ref object of tt.O
    x: float

proc finn[T](o: T) =
  echo "finalize O"

proc main =
  var oo: OO
  new(oo, finn)

main()

We import module tt.nim and subclass the ref object type tt.O. While module tt.nim defines a generic finalizer proc fin(), we can not use that one for our subclassed type OO, but have to copy it from module tt.nim into our main module, and we may have even to use a different proc name. Otherwise, we get the compiler message

Error: type bound operation `fin` can be defined only in the same module with its type (OO:ObjectType)

Whenever we should really need a finalizer or a destructor, we should prefer destructors when we can compile our code with the compiler options --gc:arc or --gc:orc.

Modules

Modules are Nim’s way to divide multiple source codes into clearly separated units and hide implementation details. Nim uses a module concept, which is very similar to Modula-2 or Oberon. All the Nim standard libraries are divided into modules that collect and logically group data types and the related procedures. In some way, modules are Nim’s classes.

In Nim, each module directly corresponds to one text file. Submodules as known from Ruby, that divide a single text file into multiple modules, are not supported by Nim currently.

Every text file with Nim source code is basically a module, and that module can be imported and used by other modules. But all symbols like data types or procedures have to be exported to make them visible and usable by other modules. That is done like in Oberon by appending an asterisk character to all names that should be exported. These restricted exports allow hiding implementation details — all symbols not exported are private to that module and can be changed and improved at any time without noticing the importing module. Note, that when we append the asterisk to the name of an object to export that type, the object's fields are still hidden and can not be accessed from within the importing module. You may append an asterisk to selected field names as well, or you may provide exported getter and setter procs for the field access. A read-only export, as known from the Oberon language, is currently not possible with Nim.

We can import whole modules, that is, all symbols that are marked for export by the asterisk, or we can import only the symbols that we need by specifying their names. Let us create a module that declares a single procedure to remove all characters from a string that are not letters:

# save this textfile with name mystrops.nim
proc remNoneLetters*(s: string): string =
  result = newString(s.len)
  var pos = 0
  for c in s:
    if c in {'a' .. 'z', 'A' .. 'Z'}:
      result[pos] = c
      inc(pos)
  result.setLen(pos)

We save the above text file with our Nim source code with name mystrops.nim. Note the export marker after the proc name. We can import and use that module like

import mystrops

echo remNoneLetters("3h7.5g:8h")

When we import modules, then we put the import statement generally at the top of the importing module, that way it is easy to see what modules are imported. The imported symbols can be used in the code after the import statement. Module names should be lower case and may as other Nim symbols only contain letters, decimal digits, and the underscore character. We can import multiple modules with a single import statement when we separate the module names with commas. Starting with Nim v1.6, it is recommended to import modules from Nim’s standard library with the std prefix as in import std/math or import std/[strutils, sequtils]. Importing the same module multiple times is not a problem, and does not increase the file size of the final executable. Note that in the import statement the module names have to be used literally, so this would not work:

const strfuncs = "stringutils"
import strfuncs

Instead of importing whole modules, we can import only single symbols with the from x import y, z syntax like

from mystrops import remNoneLetters

echo remNoneLetters("3h7.5g:8h")

Both forms are an unqualified import, that is we can refer to the proc by only its name, we do not need the qualified form with module name prefix like mystrops.remNoneLetters() as long as there are no name conflicts. But whenever we want, we can use the qualified form also.

Nim programmers mostly prefer importing whole modules and using unqualified names, while that is considered a bad style in some other languages like Python. In untyped languages like Python, unqualified imports may indeed pollute the namespace and generate many name conflicts, but in statically typed languages like Nim unqualified imports seems to generate name conflicts only in very rare cases. Procedures with the same name typically have different parameter lists, so the overload resolution of the compiler can decide what proc is to be used. And when really a name conflict occurs, then the compiler will tell us, and we can easily fix it by prefixing the proc name with its module name.

For data types, constants, or enums, chances for name conflicts may not be that tiny, so we may have to use qualified names.

We can also enforce a fully qualified import in Nim by a notation like

from mystrops import nil

In this case, we can use all symbols from that module only in qualified form. But that does not always work that well in Nim, as Nim has no classes like Java, so a qualified use of method call syntax or qualified use of user-defined or overloaded operators is difficult. Imagine strutils.add(s, '\n'), how should that look with method call syntax?

For imports, we have also the except keyword, so we may do something like

import strutils except toUpper

The except keyword can be used to prevent possible name conflicts, without having to use qualified names.

Note that the system module is imported automatically, we should not import it directly. Also note, that Nim always imports only what is really needed in the final executable, so importing only a few symbols from a module has no code size benefit over importing the whole module. Still, it may improve the readability of your code, when you import only single symbols for the case that you are sure that you require not more. Maybe like from math import Pi. Note that you can even in that case access other symbols of that module by fully qualified names like math.sin().

With the growing standard library, it may occur that module names of the standard library interfere with your own module names. So Nim now allows and recommends qualified import of modules from the standard library like import std/strutils. And for external packages installed by the nimble package manager, imports in the form import package/[mod1, mod2, mod3] are permitted.

Finally, you can also import modules under another name using the as keyword like

import tables as maps

With the recent Nim compiler, you can also enforce full qualified import and use of an alternate module name by using an import statement like

from tables as maps import nil

With this import statement, you could access symbols from the tables module only by use of the maps module prefix like maps.newTable().

Finally, with the export keyword, one library module can export other modules, which it imports itself. This may simplify the use of connected modules. As an example, when using the gintro bindings for GTK4, we import all the needed modules generally like import gintro/[glib, gobject, gtk4]. We may decide to simplify that import statement by creating one more module called gtkplus that consists only of these two lines:

# module gtkplus
import gintro/[glib, gobject, gtk4]
export glib, gobject, gtk4

Then a user of gintro could just write import gtkplus to have access to all the modules. Actually, for GTK this is not really a good idea, we will tell you more about the gintro module and maybe about one more of Nim’s GUI libraries in the second half of the book.

Cyclic Imports

Typically, we try to arrange our own modules in a tree-like bottom-up structure. A module x may define basic types and simple functions working with these types, and a higher level module y may import all symbols from module x and extend the functionality. But in rare cases, it may be necessary that the modules x and y import each other, as x has to use types or functions of module y, and vice versa. This case is called cyclic import and is currently not supported by Nim. Indeed, we should generally try to avoid cyclic imports when possible, as cyclic imports make the software design difficult. But sometimes we can not really avoid these cycles. In that case, currently, the best solution is, to put all the concerned data types in a separate low-level module, which is then imported from both other modules. The planned Nim version 2.0 may allow cyclic imports, so this restriction may vanish in the future.

Include

The include statement should be not confused with the import statement. Include just insert a text file at the position where the include statement occurs. Include can be used to split very large modules into smaller entities.

Part III: Nim’s Standard Library

In this part of the book, we will introduce you to some of the most essential modules of Nim’s standard library. This includes modules for common operations like the serialization of Nim data types to write them to external nonvolatile storage and read them back into the program later, or handling command-line options and parameters for programs launched from within a terminal window. Further, we will introduce you to important container data types like hash tables (sometimes called hash maps in other programming languages) and various kinds of set data types. We will also introduce modules for working with regular expressions, and we will show how simple modules like the times or the random module can be used. Most modules mentioned in this part will be from the Nim standard library so that you will not have to install external packages to use them. But there may be some exceptions, e.g. for some external Nimble packages with very useful functionality and an easy user interface. One of these exceptions is the regex module: Nim’s standard library comes with the re and nre modules, which both use the PCRE C library. We have decided to introduce the regex module instead, which is an external package written completely in Nim language.

Formally, Nim distinguishes between pure and impure libraries and wrappers. The majority of Nim’s standard libraries consist of pure libraries, that are modules completely written in Nim code. Impure libraries provide a high-level Nim interface, and can be used like pure libraries, but use C libraries under the hood. Examples are the two modules re and nre, which both use the PCRE C library and some database modules. Impure libraries can be used as the pure ones when the underlying C library is installed. The few wrappers that are shipped with Nim provide only a low-level interface to C libraries, which may use unsafe pointers as proc parameters and may require the user to do manual memory management. Some impure modules uses these wrappers and hide all the ugly stuff for us, but we generally do not use the wrappers directly.

Nim’s standard library is supported by thousands of external packages, which can easily be installed with Nim’s package managers, and then can be used in the same way as the modules of the standard library. The next part of the book will introduce you to the use of external packages and presents some of the most important ones.

Command-Line Arguments

When we launch a program from inside the terminal window, we can pass it some additional parameters, e.g. the name of a file to process or option parameters to influence the behavior of the program. We have done so already when we launched the Nim compiler or maybe a text editor from inside our terminal window. Using command line parameters is convenient when we work from inside a terminal and there are parameters that we know in advance. A more interactive way to collect parameters is reading in input while the program is already running, as we did in part II of the book when processing the list of our friends. We will learn some more details about this interactive processing of input in the next section.

Nim allows processing command-line arguments in the same basic way as all C programs do, but Nim’s standard library and some external packages allow also much more advanced handling of command-line arguments. For simple cases, the C-like way is sufficient. For C programs the command line arguments are even coupled very closely to the language itself, the number of arguments and the list of parameters are the two typical parameters of the C main() function and are used in this way:

// C program expecting one command line argument
// Compile with gcc t.c
#include <stdio.h>
int main( int argc, char *argv[] ) {
  printf("Executing program %s\n", argv[0]);
  if( argc == 2 ) {
     printf("The argument supplied is %s\n", argv[1]);
  }   else if( argc > 2 ) {
     printf("Too many arguments supplied.\n");
  }
  else {
     printf("One argument expected.\n");
  }
}

Here argc is the number of available arguments, and argv is an array containing the actual arguments in form of C strings. These values are passed to each C program by the OS when the program is launched from inside a terminal. Actually, the value of argc is the number of passed arguments plus one, that is when we specify no arguments at all, argc has the value of one. And argv[0] is always the name of the executed program. We have to know, that command-line arguments passed to a program are separated by white space, that is at least, by one space or tab character. For this reason, we have to enclose single arguments containing white space in double-quotes:

$ gcc t.c -o t
$ ./t Nim two
Executing program ./t
Too many arguments supplied.
$ ./t "Nim two"
Executing program ./a.out
The argument supplied is Nim two

In Nim, we have the same functionality available by use of the paramCount() and paramStr() procs, which we have to import from the os module. But paramCount() gives us the actual number of parameters, so when we call our program on the command line without any arguments, paramCount() will return the value zero. The symbol paramStr() is not a global array variable, but a procedure. ParamStr(0) gives us the name of our executable, and with arguments greater than zero we get the passed arguments as strings in ascending order. Using an index number for an argument that was not provided will cause paramStr() to raise an exception.

An argument evaluation similar to our C program from above may look like

from os import paramCount, paramStr

proc main =
  echo "Executing program ", paramStr(0)
  let argc = paramCount() + 1
  if argc == 2:
    echo "The argument supplied is ", paramStr(1)
    if paramStr(1) in ["-d", "--debug"]:
      echo "Running in debug mode"
  elif argc > 2:
    echo "Too many arguments supplied."
  else:
    echo "One argument expected."

main()

Using this plain API is OK when we expect one or two arguments, maybe a file name and an option, like the -d or --debug parameter used in the code above. For more command-line arguments, things get complicated fast, as arguments can be passed in arbitrary orders and combinations. So you should try one of the available libraries for that case.

References:

Reading data from the terminal

While using command-line arguments is convenient for data like filenames or options that we already know when we launch a program from the terminal window, often we have to provide textual user input while the program is already running. Functions for this task are provided by the io module, which is part of the system module, and which we have not to import explicitly. In one of the introducing sections of the book, we used already the readLine() and the getch() procs for reading in a line of text from the terminal and for waiting on a single keypress event.

For input and output operations in a terminal window, the io module defines the three variables stdin, stdout, and stderr of the File data type. Many procs of the io module expect as the first parameter a variable of File type. We can explicitly open a named file to write data to external media like the SSD, or we can just use the stdin and stdout variables to read data from the keyboard and to write text to the terminal window. Unlike other named files, we do not have to call open() or close() on stdin and stdout to open or close the files, and some other file operations like setFilePos() may not work for these file variables:

var s: string = stdin.readLine()
stdout.write(s)
stdout.flushFile

We mentioned already, that the readline() function lets you type textual user input, including spaces, and that you have to terminate your input by pressing the return key to pass the input string to the OS, which forwards the input to our program. This form of input is sometimes called blocking, as for the time that we wait for user input, our program is really waiting, it can not do other work till the user has pressed the return key. For single-character input, without the need for pressing actually the return key, e.g. for a simple yes/no input, you may use the getch() function, which is also blocking. In a later section of the book, we may show how we can use threading to actually do some useful work, while we wait for user input. In the literature stdin, stdout and stderr are often called streams, where stderr can be used instead of stdout for writing error messages. This can be useful in special cases when we have an application where we want to redirect error messages to a file or to separate regular output and error messages. For more details about these stream or file variables, and the use of the stderr variable, you may consult external literature, if you should really need that info.

The io module does not provide read() functions for other basic data types like numeric or boolean types. So we should use readLine() to read the user input in string form, which we can convert by functions like parseInt(), parseFloat(), or similar functions to numeric data. Note that parsing procs like parseInt() are provided by the module strutils as well as by the module parseutils — one function raises an exception for invalid input, while the other one returns a boolean value indicating conversion success. Of course, we should handle textual user input always carefully and never just assume that the input is actually valid data. Some of the modules that can be used for converting textual input data into other data types like the strutils, strscans modules are described in more detail at the end of this part of the book.

For advanced user input processing, like cursor movement, colored display, or displaying progress bars, you may also consult the terminal module. And finally, to create fancy textual user interfaces (TUIs) we recommend trying external packages like the illwill library.

References:

Writing text to the terminal window

In previous sections, we have used the echo() function to write variables of various data types to the terminal window. The echo() function accepts multiple arguments, writes the string representation of the passed arguments to the terminal window, and terminates the action by writing the \n character to move the cursor to the beginning of the next line in the terminal window. We have already used the write() function from the io module for the case that we want to write a single string to the terminal without a terminating newline character. The io module contains some overloaded write() functions for other basic data types like int, float, or bool, and a variant with a varargs parameter and applied stringify operator, so that write() can be used like echo, as long as we pass stdout as the first parameter. For the actual output operation, the C library function fprintf() is used. Note that write operations to stdout are generally buffered, so the result of write() operations may remain invisible until we write a string containing a newline character or until we call the flushfile() function to enforce the writing of the buffer.

Option types

Option types can be used to encapsulate values in a way that allows marking the value as undefined. This can be especially useful for the return types of functions, which may or may not return a meaningful value.

Let us assume that we have a function called find() that searches in a string for the first index position of a character:

proc find(s: string; c: char): int =
  result = -1 # not found
  var i = 0
  while i < s.len:
    if s[i] == c:
      return i
    inc(i)

echo "Nim".find('i')

The function returns the index position or -1 to indicate that the character has not been found. This works, as we use in Nim typically signed integers, and the valid string index positions are never negative, so it is obvious that a negative result is some form of error indication. In a similar way, whenever a function should return a reference or a pointer, the special value nil can be used to indicate the absence of a value. Actually, in most cases we can just define a special value as the indication for the absence of a result or as an error indicator, for example, int.low, char(0), or NaN for float results.

Other ways to indicate failures are to return a boolean value for success and to return the actual result value as a var parameter, to return a tuple, which encloses a boolean for success indication and the actual result, or to return the result(s) as a sequence which can be empty in case of no success:

proc find(s: string; c: char; pos: var int): bool =
  pos = 0
  while pos < s.len:
    if s[pos] == c:
      return true
    inc(pos)

var p: int
echo "Nim".find('i', p), ": ", p
proc find(s: string; c: char): tuple[succ: bool, pos: int] =
  var i = 0
  while i < s.len:
    if s[i] == c:
      return (true, i)
    inc(i)

echo "Nim".find('i')
proc find(s: string; c: char): seq[int] =
  var i = 0
  while i < s.len:
    if s[i] == c:
      result.add(i)
    inc(i)

echo "Nim".find('i')

For a more formalized way to indicate the absence of a meaningful result, many modern programming languages provide the concept of Option types, which are also called sometimes Maybe types. Option types can encapsulate an arbitrary data type and provide functions like isSome() or isNone() to test for the existence of a valid value, and functions like get() to extract the actual value from the Option type:

import std/options

proc find(s: string; c: char): Option[int] =
  var i = 0
  while i < s.len:
    if s[i] == c:
      return some(i)
    inc(i)

var res = "Nim".find('i')
if res.isSome:
  echo res.get

The options module of Nim’s standard library provides the generic Option[] data type and the functions some(), isSome(), and isNone() to create a new Option type encapsulating some data, and to check if data is present. In the code above, we use some(i) to wrap the integer value in the Option type when we have found a match. For no match, the proc returns the default empty Option type instance. When we use the find() function with the Option[int] result, we have to call first isSome() to check if valid data is available, and then call get() to get the actual data.

Nim’s Option types are based on objects. The generic Option[T] type is an object with two fields, a boolean indicating the presence of data, and a field that can store the actual data. Nim uses also an optimization: When the data type is of ref or pointer type, then the bool field is not necessary, as the absence of data is equivalent to a data entity with nil value.

The overhead of Option types is not that big — a proc which would return a 4-byte integer would return an object instead — the additional boolean field would increase the size of the result to 5 bytes, which is generally extended by the compiler to multiples of the word size, that is 8 byte total. So we have 100 % size overhead for the worst case. And the loss of performance for the encapsulation of data in Option types should be not significant in most use cases.

The options module provides some more procedures for the handling of Option types, but this short introduction should be enough to get you started.

References:

Serialization — storing data permanently on external storage

When you start writing larger programs, these programs may create data that you may want to store permanently on external nonvolatile storage like SSDs or traditional hard disks of your computer. For textual data, this is very easy, as you only have to write and read a stream of unstructured bytes. But when your program deals with object instances, container data types like sequences, or references, things become more complicated. Writing the data is always easy — you can just convert all the fields of your object data type to strings and write them to a stream or a file. But the reading back part is much more difficult: You would have to read in the data as strings, and then process each string — maybe converting it to a float number — and then assign it to the matching field of an object instance.

When your data consists only of value objects and no references, then you may consider just writing that data in plain binary form to a file and reading it back. This strategy seems to be simple, and it is very fast, as no type conversion steps are involved. But at the same time, it has some drawbacks: The stored data can not be checked with tools like a text editor, it can generally not be used from other programs, and when you should change the data types used in your program, you could not read back stored files anymore.

So we will explain how you can store Nim data types in a human-readable text format first. Two popular text formats are JSON and YAML. JSON is a simple format, which is easy to parse, but not very good readable for humans. YAML is more complicated, but more flexible and is very good readable for humans.

For Nim, we have already many modules available, which we can use for storing data in JSON or YAML format. The Nim standard library includes the marshal and the json module. The marshal module uses, like the json module, the JSON data file format, which is easy to use and simple but is not really designed to generate human-readable data files, as the stored data is not stored as a sequence of individual lines. So we will describe and use the json module in this section, which is also easy to use, but has a larger set of functionality and can generate real human-readable text files by use of the pretty() function.

Other available external packages for data serialization are the nim-serialization module set from (https://github.com/status-im/nim-serialization) and the very powerful but complicated NimYaml implementation (https://nimyaml.org/). We may describe these packages in part V of the book.

When we have to store and read back Nim data to nonvolatile storage media, we have some serious points to consider: First, we have to handle various data types like integers, floats, strings, objects — and even the container types like sequences. And we may have to support reference types and maybe also inherited types and containers filled with heterogeneous, subclassed reference objects. The json module supports all Nim data types, including containers and references, but not heterogeneous sequences.

For our first JSON example, let us assume that we have written a small tool that let the user create some geometrical shapes, and we want to store the shapes in a file and read it back. For that, we generally use an intermediate step, which converts the data to a string, and the string back to the data object. The string is then written to a file or stream, and read back. Let us start with the string conversion — storing that string and reading it back from the file will be explained soon.

import json

type
  Line = object
    x1, y1, x2, y2: float

  Circ = ref object of RootRef
    x0, y0: float
    radius: float

  Data = object
    lines: seq[Line]
    circs: seq[Circ]

var
  l1 = Line(x1: 0, y1: 0, x2: 5, y2: 10)
  c1 = Circ(x0: 7, y0: 3, radius: 20)
  d1, d2: Data

d1.lines.add(l1)
d1.circs.add(c1)
d1.lines.add(Line(x1: 3, y1: 2, x2: 7, y2: 9))
d1.circs.add(Circ(x0: 9, y0: 7, radius: 2))

let str1 = pretty(%* d1) # convert the content of variable d1 to a [.str]#string#
echo str1 # let us see how the [.str]#strings# looks
d2 = to(parseJson(str1), Data) # read the [.str]#string# back into a data instance
let str2 = pretty(%* d2) # and verify that we got back the original content
echo str2

# assert d1 == d2 would fail
assert str1 == str2

When we run the program, we would get this output:

{
  "lines": [
    {
      "x1": 0.0,
      "y1": 0.0,
      "x2": 5.0,
      "y2": 10.0
    },
    {
      "x1": 3.0,
      "y1": 2.0,
      "x2": 7.0,
      "y2": 9.0
    }
  ],
  "circs": [
    {
      "x0": 7.0,
      "y0": 3.0,
      "radius": 20.0
    },
    {
      "x0": 9.0,
      "y0": 7.0,
      "radius": 2.0
    }
  ]
}
{
  "lines": [
    {
      "x1": 0.0,
      "y1": 0.0,
      "x2": 5.0,
      "y2": 10.0
    },
    {
      "x1": 3.0,
      "y1": 2.0,
      "x2": 7.0,
      "y2": 9.0
    }
  ],
  "circs": [
    {
      "x0": 7.0,
      "y0": 3.0,
      "radius": 20.0
    },
    {
      "x0": 9.0,
      "y0": 7.0,
      "radius": 2.0
    }
  ]
}

As you can see, we converted the instance d1 of type Data to a string, and then we convert that string back to variable d2, with matching content. We have made intentionally Circ a ref object, so we can see that the conversion works for value and reference objects. In the example program, we applied the %* macro to our data instance d1 to get a JsonNode, and finally use the pretty() function to get a nice multi-line string. To fill the variable d2 with the content stored in str1, we first have to apply parseJson() on the string, and then use to() to unmarshal the JSON node into the matching object type.

Now, let us investigate what happens when we try to use the json module with a container with heterogeneous ref objects. For that, we subclass the Disc type, creating a new Arc type:

import json
from math import PI

type
  Line = object
    x1, y1, x2, y2: float

  Circ = ref object of RootRef
    x0, y0: float
    radius: float

  Arc = ref object of Circ
    startAngle, endAngle: float

  Data = object
    lines: seq[Line]
    circs: seq[Circ]

var
  d1, d2: Data

d1.lines.add(Line(x1: 0, y1: 0, x2: 5, y2: 10))
d1.circs.add(Circ(x0: 7, y0: 3, radius: 20))
d1.lines.add(Line(x1: 3, y1: 2, x2: 7, y2: 9))
d1.circs.add(Arc(x0: 9, y0: 7, radius: 2, startAngle: 0, endAngle: PI))

echo d1.circs[1] of Arc, " ", Arc(d1.circs[1]).endAngle

let str1 = pretty(%* d1)
d2 = to(parseJson(str1), Data)
let str2 = pretty(%* d2)
echo str2
echo d2.circs[1] of Arc

The output of that program looks like this:

true 3.141592653589793
{
  "lines": [
    {
      "x1": 0.0,
      "y1": 0.0,
      "x2": 5.0,
      "y2": 10.0
    },
    {
      "x1": 3.0,
      "y1": 2.0,
      "x2": 7.0,
      "y2": 9.0
    }
  ],
  "circs": [
    {
      "x0": 7.0,
      "y0": 3.0,
      "radius": 20.0
    },
    {
      "x0": 9.0,
      "y0": 7.0,
      "radius": 2.0
    }
  ]
}
false

While our initial instance d1 contains a run-time value of Arc type, and so we can access the endAngle field, we get false as result for the of Arc test for the d2 instance. So run-time type information is lost.

When we have to store different data types in one container, then one solution is to use object variants, which should work with the json module. Another obvious possibility is to just copy the data into containers with the appropriate static type before storing in an external medium, and copy them back when we read the data back from external storage. Will will show an example of that now:

import json
from math import PI

type
  Line = ref object of RootRef
    x1, y1, x2, y2: float

  Circ = ref object of RootRef
    x0, y0: float
    radius: float

  Arc = ref object of Circ
    startAngle, endAngle: float

  Data = object
    elements: seq[RootRef]

  Storage = object
    lines: seq[Line]
    circs: seq[Circ]
    arcs: seq[Arc]

const
  DataFileName = "MyJsonTest.json"

var
  d1, d2: Data
  storage1, storage2: Storage
  outFile, inFile: File

d1.elements.add(Line(x1: 0, y1: 0, x2: 5, y2: 10))
d1.elements.add(Circ(x0: 7, y0: 3, radius: 20))
d1.elements.add(Line(x1: 3, y1: 2, x2: 7, y2: 9))
d1.elements.add(Arc(x0: 9, y0: 7, radius: 2, startAngle: 0, endAngle: PI))

for el in d1.elements:
  if el of Arc:
    storage1.arcs.add(Arc(el))
  elif el of Circ:
    storage1.circs.add(Circ(el))
  elif el of Line:
    storage1.lines.add(Line(el))
  else:
    assert(false)

let str1 = pretty(%* storage1)

if not open(outFile, DataFilename, fmWrite):
  echo "Could not open file for storing data"
  quit()
outFile.write(str1)
outFile.close

if not open(inFile, DataFilename, fmRead):
  echo "Could not open file for recovering data"
  quit()
let str2 = inFile.readAll()
inFile.close

assert str1 == str2

storage2 = to(parseJson(str2), Storage)

for el in storage2.lines:
  d2.elements.add(el)
for el in storage2.circs:
  d2.elements.add(el)
for el in storage2.arcs:
  d2.elements.add(el)

for el in d2.elements:
  if el of Arc:
    echo "found arc with endAngle: ", Arc(el).endAngle

For this example program, we use the object-orientated programming style and keep all the geometric object instances as references in a single sequence. Note that doing this is not always a good idea, as this OOP style with the use of references and dynamic run-time dispatch can be slower due to many small heap allocations for each ref object and due to the dynamic dispatch (if el of …​) overhead. Using multiple, homogeneous sequences with value types for each of our data types can be a better solution, and in that way, you have more control whenever you process the data, for drawing them on the screen or user interaction for example. Maybe you want to draw all the lines first? But there can be situations where we really need to have all the objects as references in a single container. A typical situation is, that we use an RTree for fast object location. RTrees are data structures, that can store two-dimensional or multi-dimensional geometric objects and their rectangular bounding boxes in a tree-like fashion for fast object location. This may be used in a drawing program so that coordinates of a user’s mouse click can be fast matched to an object. For such a use case, we would really prefer to have all the object instances available in one single RTree, and not use one RTree data structure for each object shape.

Our program defines an additional Storage data type, which contains homogeneous sequences for each possible geometric shape. We then copy all our ref objects from the elements sequence in the matching sequences of the storage object using the dynamic of type query to select the exactly matching sequence.

After that, we can use the already known JSON functions to serialize the storage object into a string, store the string to a file, read it back, and deserialize the data again into a different variable of Storage data type. Finally, we use a simple for loop to copy the ref objects from the temporary storage object into a Data variable called d2. For storing the data in an external nonvolatile medium, we use the File data type and the related functions open(), close, write() and read(). Their use should be obvious: We pass an uninitialized variable of File data type, a file name, and a file mode to open(), use write() to write the whole string, and use readAll() to read the data back. When done with each file, we use close() to close the file. The File data type is part of the io module, which is again part of the system module, so we don’t have to import these modules. We could have used as an alternative also the streams module. We will learn some more details about the File data type and the streams module in later sections of the book.

We should mention that unfortunately, life is not always that easy, as sometimes we can not freely select the textual output format. Imagine that you create a CAD (computer-aided design) tool that should be compatible with another existing tool. In this case, the textual storage format is already defined by the existing tool, and generally, that format does not match the JSON or YAML file format. Even when the format should be one of these, matching it exactly would be difficult. While writing out our own data in that foreign format is still not really difficult, as we can just write single matching strings, reading in the textual data is more complicated: Typically, we would read the input file line by line, and we would have to inspect and interpret each input string, maybe by the use of regular expressions or a custom parser. That generally includes handling missing or invalid data.

References:

Streams and Files

In the previous section, we learned how we can store structured data like a sequence of objects, in a human-readable form to nonvolatile media by use of the json module.

Text in form of a single string, or in form of a container holding multiple strings, is some kind of unstructured data, which we can write directly to nonvolatile storage, and later read it back. We can do the same with containers of basic, unstructured data types like integer or floating-point numbers, and with some restrictions, we can even write tuples or objects directly as raw bits and bytes to external storage and read them back later. Of course, this way the stored data is a binary blob, which can not be read or modified by other tools, like a text editor. But that may not be intended or advantageous at all, maybe we do scientific data processing with a single tool, and we just want to temporarily store the data and continue with the processing later.

Files

For storing unstructured data, Nim provides the io module with the File data type and related procs, and the streams module with the Stream data type and related procs. While a File in Nim is currently only a pointer to a C file, the streams module has a higher abstraction level. Although the Nim language does not directly support interfaces, the Stream data type of the streams module is some form of an interface, which is implemented by a StringStream and a FileStream data type. Internally, this interface concept is realized by storing a set of function pointers in the Stream instance.

When we have to store unstructured data like text, it is not always clear if we better should use Files or Streams. Streams may be the better choice when we (also) want to use a string as a data source like a file or when we need the peek() functions of the streams module to access data without advancing the position in the stream.

We will use the File data type of the io module first. As the io module is part of the system module, we do not have to import it before we can use it. The principle usage of files is, that we call the function open() to open a file with the given name, call some procs to write or read data, and finally close() the file. While Nim supports destructors, when we compile with --gc:arc or --gc:orc, the io module does not yet use them, so we should actually call close() to close the file.

Historically, a file is a one-dimensional data type, which is accessed in sequential order. Up to the end of the twenty century, it was not uncommon that large files were stored on magnetic tapes, which could be read or written only slowly in sequential order. Read or write operations could take place only at the actual position, and available functions like f.setFilePos() were very slow as it involves moving the tape. The introduction of hard disks and solid-state disks removed this restriction, and modern operating systems often buffer files in RAM for longer time periods, so that files may have actually similar performance as arrays or sequences. The funny fact is, that with the modern CPU caches, the ordinary RAM storage can look similar slow and sequential compared to the extremely fast cache, as magnetic tapes in the past.

from os import fileExists
proc main =
  const FN = "NoImportantData"
  if os.fileExists(FN):
    echo "File exists, we may overwrite important data"
    quit()
  var f: File = open(FN, fmWrite)
  f.write("Hello ")
  f.writeLine("World!")
  f.writeLine(3.1415)
  f.close
main()

Running that program will create a text file with this content in the current working directory:

Hello World!
3.1415

At the start of our function, we check if a file with that name already exists in the current working directory by using the function os.fileExists() to ensure that we do not overwrite important data.

Module io provides multiple overloaded open() procs. We use here a variant that returns a file and raises an exception for the unlikely case of an error. We provide a file name and a file mode as parameters. We use mode fmWrite, as we want to create a new file. Note that fmWrite would clear the content of an existing file, so we can not use fmWrite to append data to an existing file. We would have to use fmReadWriteExisting or fmAppend to append data to an already existing file. As this open() proc can raise an exception, it may make sense to enclose it in a try/except block, or we could use an open() variant which returns a boolean value to indicate success instead. When the file is successfully opened, we can use procs like write() or writeLine() to write text strings to the file. Both procs accept multiple arguments and apply the stringify operator $ on them before writing the content. WriteLine() writes a '\n' after the last argument to start a new line. When done, we call close() to close the file. The operating system would close the file for us when our program terminates, so calling close is not that important, but when we open many files without closing them, we may get errors from the operating system finally about too many open files and our program may fail or terminate.

The close() proc gets passed the file not as a var parameter, so it can not set the file to value nil. When the file has the value nil, then the close() call is ignored, but when we would call close() multiple times with a non-nil argument, we get a program crash. We may use the try/finally or the defer construct to ensure that we really close the file when done.

The io module provides some procs like writeBuffer(), writeBytes(), or writeChars(), which gives us as a return value the actual number of bytes written. This return value should generally match the requested number of bytes to write, but can be smaller when the write operation fully or partially failed, e.g. because the storage medium had no capacity left.

When performance really matters, we should note that passing non-string arguments to write() or writLine() procs using their optional auto-stringify for us, involves the allocation of new strings and cost some performance. When we have in our program already a string variable available, it can be faster to convert our data into that variable first and then pass that variable to the write() or writeLine() procs.

Reading strings from a file works very similar:

proc main =
  var f: File
  try:
    f = open("NoImportantData", fmRead)
    echo f.readLine
    echo f.readLine
  finally:
    if f != nil: # test for nil not really necessary, close() would ignore the call for f == nil
      f.close
main()

The readLine() proc reads in a line of text. The LF, CR, or CRLF line end markers are not part of the returned text string. Of course, we may get an empty string with length zero back, when we read a line that immediately starts with LF, CR, or CRLF, or we may get back a string with no visible characters but only a few spaces or tabulator characters '\t' when a line contains only white space. When our read() operations have moved the actual file io position to the end of the file, and we try to read more content, then an exception is raised.

The io module provides a readLine() proc that returns a newly allocated string and one that takes an existing string as a var parameter. The latter may be a bit faster, as it can avoid the allocation of a new buffer when the passed string has already enough capacity.

The io module provides a function called endOfFile() with a boolean result, which we can use to check if the end of file position is already reached. The provided functions readBuffer(), readBytes(), or readChars() return the actual number of bytes read, which can be smaller than the requested value when the end of the file is reached earlier. Currently, readChars() checks if the passed openArray[char] has enough capacity for the request, but readBytes() does no check!

We can also use the lines() iterator to iterate over the lines of a text file or use the readLines() proc to read the content line by line.

proc main =
  var f: File
  f = open("NoImportantData", fmRead)
  for str in f.lines: # iterator
    echo str
  f.setFilePos(0) # read again from start index 0
  var s: string
  while f.readLine(s): # proc
    echo s
  f.close
  var sq = readLines("NoImportantData", 2) # read lines to seq of strings
  echo sq
main()

As iterating over the complete file line by line moves the actual file position to the end of the file, we called setFilePos() to move again to the start position. The readLines() proc takes a filename and the number of lines to read as parameters, and returns a seq of strings. When the file does not contain at least the number of requested lines, an EOF exception is raised. Another provided proc is readAll(), which reads the entire file content into a returned string variable. For readAll() to work, the actual file position has to be the start of the file. In case of an error, an exception is raised.

We can also write and read binary data directly to a file, without converting it to (human-readable) strings first:

proc main =
  var f: File
  f = open("NoImportantData", fmWrite)
  var i: int = 123
  var x: float = 3.1415
  assert f.writeBuffer(addr(x), sizeof(x)) == sizeof(x)
  assert f.writeBuffer(addr(i), sizeof(i)) == sizeof(i)
  f.close
  f = open("NoImportantData", fmRead)
  assert f.readBuffer(addr(x), sizeof(x)) == sizeof(x)
  assert f.readBuffer(addr(i), sizeof(i)) == sizeof(i)
  f.close
  echo i, " ", x
main()

Of course, these are low-level, dangerous operations. While writeBuffer() should never crash our program, readBuffer() can do that easily when we specify wrong sizes or destination addresses, as that may overwrite other data unintentionally. So we would generally not use these procs directly but write more safe helper procs, when we really need or want this form of binary file access. Fast storing big data sets with restricted hardware may be a use case, e.g. storing a float32 takes only 4 bytes on the storage medium and file io is fast, while that number as human-readable digits may need more than 8 bytes (1.234567E3) and converting to string and parsing back costs some time.

In the same way, we can use writeBuffer() and readBuffer() to store tuples, objects, and arrays or sequences of these directly in binary form:

type
  O = object
    x: float
    i: int
    b: bool

proc main =
  var s: seq[O]
  s.add(O(x: 3.1415, i: 12, b: true))
  var f: File
  f = open("NoImportantData", fmWrite)
  assert f.writeBuffer(addr(s[0]), sizeof(O) * s.len) == sizeof(O) * s.len
  f.close
  f = open("NoImportantData", fmRead)
  var s2 = newSeq[O](1)
  assert f.readBuffer(addr(s2[0]), sizeof(O) * s2.len) == sizeof(O) * s2.len
  f.close
  echo s2[0]
main()

The output should look like

(x: 3.1415, i: 12, b: true)

But of course, this is dangerous and fragile. We just show that example, as beginners typically ask about it, and may want to try it at least once. Obviously, this can only work, when the tuples or objects contain only plain data types, that is no string, no references, and of course no other nested container types like sequences or tables. And reading back data may fail when we use a different OS or a different compiler version.

The io module provides the File variables stdin, stdout, and stderr, which are the standard input, output, and error streams. Sometimes we use stdout.write() instead of the common echo() proc when we want to write something to the terminal window without moving the cursor to the next line already.

An important function of the io module is flushFile(), which is used to ensure that all buffer content of buffered files is actually written to the file. This is important when we use the stdout File variable, maybe to ask the user a question in the terminal window. We would call sdtout.flushFile() to ensure that the user really sees the text on the screen immediately. The echo() proc calls flushFile() automatically after each output operation. When we close a file, flushFile() should be called automatically, but when our program is terminated without calling close(), it may depend on the actual implementation and operating system.

The io module provides some more useful procedures, but we will stop this introducing section here, and continue with the streams module in the next section.

References:

Streams

A stream is an abstract interface for performing certain I/O operations, which was introduced by languages like C or Modula-2 decades ago. The streams module of the Nim standard library provides a FileStream and a StringStream implementation, which behave very similarly. Nim’s streams module provides similar functions as the io module with its File data type, but it can operate on strings instead of on Files, and it provides a set of peek() functions to access data at the current read position without moving forward. And some functions are more robust, for example, closing a stream multiple times does not crash the program, as the first close() call sets the file variable of file streams to nil, so that following close() calls are ignored. Currently, the streams module does not support automatically closing streams when they go out of scope.

We can create a new FileStream by calling the overloaded procs newFileStream() with an already opened file or a filename as parameter, or we can use openFileStream(). The latter raises an exception when the stream can not be opened, while the former procs just return nil. We can write and read textual data with the streams module in a very similar way as we did it with the io module and the File data type:

from os import fileExists
import streams

proc main =
  const SN = "NoImportantData" # stream name
  if os.fileExists(SN):
    echo "File exists, we may overwrite important data"
    quit()
  var fstream = newFileStream(SN, fmReadWrite)
  if fstream != nil:
    fstream.write(123, ' ')
    fstream.writeLine(3.1415)
    fstream.setPosition(0)
    let l = fstream.readLine()
    fstream.close()
    assert l == "123 3.1415"
main()

We test again if a file with that name already exists. Then we try to create a new FileStream by using file mode fmReadWrite, so that we can write and read from that file. Finally, we write two numbers, which are automatically converted to strings, set the file position back to the beginning, and verify what we wrote by reading it again before we close the stream.

In a very similar way, we can write to and read from string streams:

import streams
proc main =
  var stream = newStringStream()
  stream.write(123, ' ')
  stream.writeLine(3.1415)
  stream.setPosition(0)
  let l = stream.readLine()
  stream.close()
  assert l == "123 3.1415"
main()

In the example above, we do not test if the stream variable is not nil, as newStringStream() should never fail.

For buffered streams, we can call flush() to ensure that the buffer content (of file streams) is written, similar to what we can do for plain Files of the io module. Instead of io.endofFile(), we use the proc atEnd(), to test if the current stream position is already at the end of the stream. Functions getPosition() and setPosition() are available to query or set the actual position in the stream. While the io module with its File data type supports for io.setFilePos() although position modes relative to the actual position or relative to the file end, streams.setPosition() always use absolute values, that is positions measured from the beginning of the stream. The streams module also provides the low-level procs readData() and readDataStr(), which read data to a memory region or into a string and returns the actual number of bytes read to indicate success. And as for the io module, a proc readAll() is available to read all data of a stream into the returned string variable.

The procs writeLine() writes the passed arguments always as strings. The overloaded write() procs with varargs arguments write the passed values as strings and apply the stringify operator $ if necessary. The same does the writeLine() proc, but it writes a newline character when all passed variables have been written. One more overloaded write proc for single string parameters exists.

But for single non-string arguments, a generic write() proc is used, which writes numbers (and other data types like boolean types or single characters) directly in binary form without converting them to strings.

To read the binary numbers back, we can use functions like readFloat64() which have a well-defined return type and read a fixed number of bytes. Or we can use the generic read() proc, which accepts a var parameter that defines the data type that we intend to read in binary form. Additional to the various read() procs, the streams module provides a set of peek() procs, which reads data without moving the actual position in the stream forward. This may be useful for parsing files, as we can read the same information multiple times easily. Internally, the peek() function uses a call of setPosition() to save the current position and one more call of setPosition() to set back the old position to the initial value, so peek() has some overhead.

import streams
from os import getFileSize
proc main =
  const SN = "NoImportantData" # stream name
  var fstream = newFileStream(SN, fmReadWrite)
  if fstream != nil:
    fstream.write("012") # write a 3 byte string
    var pi: float64 = 3.1415
    fstream.write(pi) # write as 8 byte binary blob
    fstream.setPosition(0) # prepare for reading from start
    var i16: int16
    i16 = fstream.peekint16 # read first 2 bytes as int16, do not change actual position
    assert i16 == '0'.ord + '1'.ord * 256
    var i8: int8
    i8 = fstream.readInt8 # read back one byte
    # fstream.read(i8) # does work also
    assert i8 == '0'.ord # char(0) has position 48 in ASCII table
    assert i8 == 48
    var buffer: array[2, char]
    fstream.read(buffer)
    let x = fstream.readFloat64 # read back in binary form
    assert x == 3.1415
    fstream.close()
    assert buffer == ['1', '2']
    assert os.getFileSize(SN) == 3 + 8 # 3 byte string and a float64
main()

In the example above, we write a three-byte long string and a float64 to the file stream. We call setPosition(0), to read the stream from the beginning again, and then read in an int16 with the function peekint16() without moving the actual position forward, followed by readInt8(), which moves the actual position one byte forward. (Instead of readInt8(), we could also call read() with variable i8 as passed var parameter.) Then we read in two bytes, and finally the float64 value at the end of the stream. Finally, we check, by use of the function getFileSize() from the os module, if the file has really the expected size.

The streams module provides many functions, and the possible writing of data as strings or in binary form can make using that module a bit daunting at first. But most procs have examples in the API docs, which helps you to use them.

For reading strings and whole lines, the streams module provides functions like readLine(), peekLine(), readStr(), and peekStr(), each in a variant that returns a newly allocated string, and one that uses a passed var parameter to return the string. The variants with var parameters may be a bit faster, as they can avoid allocating a new string when the passed in var parameter has already enough capacity.

References:

String Processing

String processing is a wide area. Nim’s standard library provides various pure Nim modules like strutils, parseutils, and strscans for supporting this task, and the impure re and nre modules and external packages support more advanced operations like string pattern matching with regular expressions (regex) or by use of the parsing expression grammar (PEG). We will start with the strutils module, which is one of the most used modules of the Nim standard library. Then we will introduce some more specialized modules like strscans, parsecsv, parseutils and strformat. While parsing strings with regular expressions or by use of the parsing expression grammar is very flexible and powerful, it is not that easy, not the fastest solution, and not that often really needed. So we have moved the Regex section to the end of this part III of the book, and we introduce PEGs later in part V, where we discuss some useful external packages.

Whenever we do string processing in Nim, we should care a bit for performance, as some string operations can be slow by design. For simple tasks, we should prefer to use functions from the simple modules like strutils when possible, and use Regex or PEG only when really necessary or when performance is uncritical. And even when we use elementary simple functions like a string split, it is generally good to have a feeling of how the requested operations may work. Whenever string functions return a string as a result, this implies an allocation, which takes some time and consumes some memory. An example is the split() operation, which returns a sequence of multiple strings. The split() function is easy to use, so it is often the first choice when we read lines of text from files and want to process them. But as for each section of the split line a string is allocated, it may not be as fast as desired. In some cases, the compiler may be able to optimize the slitting process, but it may be also a good idea to think about other ways to extract the data, maybe by applying procs from the strscans module, which can parse lines directly into passed var parameters, avoiding unnecessary allocations. Nim 2.0 may get support for view types, which functions like split() may use as return types to reuse slices of the initial string instead of allocating new result strings.

Remember that Nim strings are value types and have value semantics. String assignment copies the string content and does not create just a reference as in some other programming languages. Nim also defines a string variant called TaintedString, which is mainly just an alias for an ordinary string, as long as the taint mode is not turned on. Functions like io.readLine() return tainted strings, which typically can be used like ordinary strings.

Basic string operations

We discussed Nim’s string data type already in part II of the book. Remember that a string in Nim is a variable size container for ASCII characters. Strings can be plain ASCII strings, or the bytes of the string can be interpreted as Unicode glyphs. Nim has also a cstring data type, which was initially introduced to be compatible with the character arrays used in C libraries as strings, but is now called compatible string as cstrings do also support the JavaScript backend. Nim strings can be passed directly to C libraries, as a Nim string contains a Null terminated buffer for the actual data, which is identical to a C language string. So whenever we convert a Nim string to a C language string or pass a Nim string to a C library, this is free of cost, while converting a C string to a Nim string always means allocating a new Nim string and copying the data content. Technically, for a Nim string s addr s[0] is the C string pointer, called * char in C language. Whenever we pass strings to C libraries, we have to care for the fact that Nim’s garbage collector may deallocate the string automatically. Most C libs create copies of passed strings when the library uses the string for a longer time span. GTK, for example, does this with text for its widgets. But when the C library does not copy the string but uses it directly for a longer time, then it can occur that the Nim code frees the string, as the only Nim variable referring to the string goes out of scope, but the C library still uses the string. For that rare case, we may call GC_ref() on the string to prevent garbage collection, but that may generate memory leaks then. In the case that C libraries create strings, they provide generally also a function to deallocate the string. When we use such a C function, it is typically the best solution, that we copy the string from the C library to a Nim string and immediately deallocate the C string by a call of the provided free()/dealloc() function. For most C libraries, there exist good high-level bindings, which do not have these issues, so we mostly can use the C libs like pure Nim libs.

Nim’s system module provides already some basic string operations, like accessing single characters by the subscript operator [], accessing slices of multiple adjacent characters, or joining multiple strings with the & operator. The overloaded add() functions to append single characters or other strings to existing string variables are also provided by the system module.

var s: string = "I like"
s &= " Nim."
s[^1] = '!'
s[0 .. 4] = "We low" # result is: "We love Nim!"

We start the above example by assigning to the string variable s a string literal, then we append one more string literal, and finally replace the last character and the first five characters with another character and by another string. Note that by using the slice operator, we can not only replace character ranges, but we can also replace slices of different lengths. This way, we can also delete ranges in the string by replacing them with the empty string "".

The system module also defines the stringify operator $, which converts expressions to the string presentation when we put it in front of it. Procedures like echo() apply the stringify operator automatically to all of its arguments when necessary. And the system module provides the contains() function, which we can use to test if a string contains a character. Instead of contains(), we can also use the in operator.

echo $7
echo "Nim".contains('i')
echo 'i' in "Nim"

The system module also provides the procs newString() and newStringOfCap(), which are mostly used for optimizing purposes. The function newString(n) creates a string of length n, but with uninitialized content. We would have to assign characters to the positions 0 .. n-1 to create a valid string. The function newStringOfCap() creates a string with a length zero, but with a buffer capacity of n characters. When we know the needed buffer capacity, or at least a lower bound of it, it makes sense to create the string with newStringOfCap() with optimal buffer size to avoid reallocations. Of course, we could still append more data, Nim would allocate a larger buffer and copy the old content.

var s1 = newString('z'.ord - 'a'.ord + 1)
for c in items('a' .. 'z'):
  s1[c.ord - 'a'.ord] = c

var s2 = newStringOfCap(32) # we intend to append not more than 32 characters, but we could do.
for c in 'a' .. 'z':
  s2.add(c)
# s1, s2 is abcdefghijklmnopqrstuvwxyz

Filling in characters into an existing string by use of the subscript operator [] is faster than appending single characters with the add() function because the add() function has to check if the string has still enough capacity and because add() has to increase the actual string length by one for each call.

Note, that a single character like 'a' is very different from a string with only one character like "a". A character in Nim is nothing more than a single byte, while a string — even one with only one character or an empty one — is an opaque entity with length, capacity, and a pointer to a data buffer. When a single character is sufficient, we should use that and not a string containing a single character. A function call like s.add("a") may produce less optimized code than s.add('a'), but maybe the compiler optimizes the former for us. When we consider optimization, we may wonder if in

var s = "Hello!"
echo s
s = "Bye."
echo s

line 3 allocates a new string or just copies the string literal "Bye." in the existing data area. Well, we would hope for the latter, of course.

Another interesting question is, if in

var s = "Result is: "
var x = 12.34
echo s, x
echo s & $x

we should better use line 3 or line 4. Line 3 looks more clear, and we assume that it also would produce better code, as the actual append operation is avoided.

Often used functions are len() and setLen() to query and set the length of a string. While len() looks like a function call, it is a compiler intern function, so calls are fully optimized. So it is OK to write

for i in 0 .. s.len():
  echo '*'

It is not necessary to introduce a temporary variable like let l = s.len() to avoid many function calls. In C this is different, as in C a call of function strLen() would not only imply a function call, but that function would really have to count all the characters up to the terminating '\x0', as C strings have no length field. The function setLen() is mostly used to truncate strings. A call like s.setLen(0) makes s look like a newly allocated string, but its data buffer can be reused. Reusing strings is generally better for performance than allocating many new strings. The setLen() function is rarely used to increase the string length — in that case, an allocation of a larger data buffer can occur, and the string would still look the same as initially, and all the new string positions would still contain the default binary zero content. We would have to fill in actual characters by use of the [] subscript operator. In Nim version 1.6, a call of s.setLen(0) overwrites the whole old content with '\0', which can cost some performance and may be avoided in version 2.0. To get the lowest and highest character index of a string, we can use s.low and s.high. s.low should always return zero, and high() is identical to len() - 1, so high() is -1 for an empty string. Note that while calling high() and len() on a Nim string has no costs, this may be different to C strings as these have no length field.

The system module provides the overloaded & operator, which we can use to concatenate chars and strings, the &= operator to append a new string to an existing string, and the add() functions to append characters or strings to an existing string. For best performance, we should try to use always the most simple, "native" operations, at least when that does not make the code ugly, or we know for sure that the compiler optimizes it for us.

var s = "Ni"
s = s & "m" # maybe not a good idea for optimal performance
s.add('m') # better

The function add() can be also used to add a cstring to a string, and the JS backend allows even to append one cstring to another one by add() calls.

The module system also exports a substr() proc, which copies and returns a slice of a string. Overloads with optional first index with default 0 and optional last index with default s.high exists.

var s = "Hello!"
assert s.substr(2, 3) == s[2 .. 3]

s = "ABC"
echo s[0 .. 5] # fail
echo s.substr(0, 5) # index is clipped to s.high
s[0 .. 5] = "" # fail

And of course, == and != operators for string comparison are provided. To test if a string is empty, we can compare it with an empty string literal or test if len() is zero. The latter is guaranteed to have the best performance, but the former is a bit shorter and should be not bad performance-wise. Some other languages provide an empty() test function for this, we may define our own when we really want.

if s != "":
if s.len != 0:

When we pass a string to a proc that does no operations with it, maybe it only calls echo() or stdout.write() to print it, then it may have a tiny performance advantage to pass it as cstring. This is similar as we may pass sequences as openArray to functions, which also avoids one level of indirection. Also note, that while a Nim string is a value type, so we can not test it for nil or return nil from a proc that shall return a string, this restriction does not hold for cstrings. Unfortunately, recent Nim versions have started to complain when we use cstrings as proc parameters. In some scenarios, e.g. when we use Nim trampoline functions to call C libraries, these complaints may not be really justified.

Module stringutils

The module stringutils provides a set of functions (120 currently) and a few iterators for simple string operations. Using that functions and iterators is simple in most cases and mostly well explained in the API docs. Remembering which functions exist, their exact name, and the function arguments can be a bit difficult at first. We will introduce in this section some of the most used or more difficult functions, and give some warnings when the actual performance may not be as good as expected.

Performance critical operations are generally that one, which has to allocate new strings or that has to shift many characters, like text inserting operations. Note that some functions of this module, like toUpperAscii() work only with the lower and upper ASCII letters. For Unicode operations, we may need the unicode module.

In this section we use the term whitespace, which refers to the invisible characters {' ', '\t', '\v', '\c', '\n', '\f'} (space, tab, vertical tab, carriage return, new line, form feed). Note that we have two possible newline characters {'\c', '\n'} that starts a new line, and that older windows text files may still use the two character string "\c\n" to start a new line. The character set {'A'..'Z', 'a'..'z'} is called (ASCII) letters, the set {'0'..'9'} (decimal) digits, and the set {'0'..'9', 'A'..'F', 'a'..'f'} hex digits, used to represent hexadecimal numbers.

The stringutils module support string interpolation by use of the % operator:

echo "The $1 programming language is $2." % ["best", "Nim"]
echo "The $# programming languages are $#." % ["most difficult", "C++, Rust and Haskell"]
echo "A $adj programming language is $lang." % ["adj", "low level", "lang", "assembly"]
echo "Let's learn $#" % "Nim!"
echo format("I know $# programming languages.", 2)

We can use $1 up to $9 to mark positions where string n from the array should be inserted or just $ to insert the strings in the order as they appear in the array. We can also use named insert markers and specify name-value string pairs in the array. For a single string, we can omit the array and pass it just as a string, and finally, we can use the format() proc to enable stringify for the parameters.

We mentioned already the useful but performance-critical set of split() functions:

import strutils
let str = "Zero, first, second, third"
var s: seq[string] = str.split(", ")
echo s
echo str.split(", ", 0)
echo str.split(", ", 1)
echo str.split(", ", 3)
echo str.split(", ", 4)
@["Zero", "first", "second", "third"]
@["Zero, first, second, third"]
@["Zero", "first, second, third"]
@["Zero", "first", "second", "third"]
@["Zero", "first", "second", "third"]

We used the split() variant, which accepts a string as a split marker. This function accepts one optional parameter for the number of splits to execute and returns a sequence containing the single strings. The split marker string, also called separator, is removed from the strings. The default value for the number of splits is -1, indicating that we want a split at each separator position. If we specify a positive number n, then only n splits are executed and the last element of the returned sequence will contain the remainder of the string. When we specify for the intended splits a value, which is larger than the number of contained split markers, then we get a full split.

The reason why this function is not very fast is, that it has to allocate a sequence for the return value and a new string for each split, and one more for the last string or the remainder. For the case that we do need only the first few strings of the split, it is a good idea to specify the number of actual splits to increase performance. Nim 2.0 may fully support view types, which may avoid all the string allocations for the returned results and so improve the performance.

The strutils module provides overloaded functions, which use single characters as separators or which accept a set of characters as separators, so we may split at space, tabulator, comma, or semicolon with {' ', '\t', ',', ';'}. And a function splitWhitespace() is available to split at whitespace-like spaces and tabulators, which removes all the whitespace between the strings. Notice that the split() function for strings or single characters does one split for each separator, so we can get empty strings as result as in

import strutils
let str = "Zero___first_second__third"
let str2 = "Zero first\tsecond           third"
echo str.split('_')
echo str2.splitWhiteSpace()
# @["Zero", "", "", "first", "second", "", "third"]
# @["Zero", "first", "second", "third"]

An interesting behavior of splitWhiteSpace() is that whitespace at the start or end of a string is just ignored, while the split() function returns additional empty strings when the string to split starts or ends with the separator:

import strutils
let str = "_Zero_first_second_third_"
let str2 = "   Zero first\tsecond           third   "
echo str.split('_')
echo str2.splitWhiteSpace()
# @["", "Zero", "first", "second", "third", ""]
# @["Zero", "first", "second", "third"]

Another function is splitLines(), which splits a string at CR, LF, or CR-LF characters.

For these splitting functions also iterator variants exist, which behave like the functions with the same name. When we limit for the iterators the number of splits to perform, we may get as the last returned value the remained string. You may consult the API docs for details if necessary.

Functions with names rsplit() are also available, which behave like split() but start the splitting process from the end of the string, so we can get the file extension of a file name with something like filename.rsplit(1)[^1].

The functions removePrefix() and removeSuffix() can sometimes help to avoid expensive split operations. There is an overloaded function, that removes all single characters, all characters from a set or a single string:

import strutils
var s = "NNimmm Language"
s.removePrefix('N'); echo s # immm Language
s.removePrefix({'N', 'i', 'm', ' '}); echo s # Language
s.removePrefix("Langu"); echo s # age
s. removeSuffix("ge"); echo s # a

Other useful functions are startsWith() and endsWith(), which accept a single character or a string and return a boolean value:

import strutils
var s = "Nim Language"
echo s.startsWith('N') # true
echo s.startsWith("Nim") # true
echo s.endsWith('e') # true
echo s.endsWith("Programming") # false

An efficient function similar to find() is continuesWith() used like "Nim Language".continuesWith("Lang", 4) which tests if a string contains a substring at position n.

We can use the join() proc to join strings in arrays or sequences to single strings with an optional glue string. Join() works also when the elements are not already strings — in that case, the stringify operator is applied first:

import strutils
var a = ["Nim", "Rust", "Julia"]
var s = a.join(", ")
echo s
echo s.split(", ").join("; ")
echo [1, 2, 3].join("; ")
# Nim, Rust, Julia
# Nim; Rust; Julia
# 1; 2; 3

The overloaded find() functions accept optional start and end positions and return the index position of the first match or -1 when the search gave no result:

import strutils
var s = "Nim Language"
echo s.find('L') # 4
echo s.find({' ', 'a'}) # 3
echo s.find("age") # 9
echo s.find("Rust") # -1

The contains() function can be used to test if a character, a set of characters, or a substring is contained in a string. Instead of contains(s, sub) we can write sub in s. Note that a function variant for single characters is defined in the system module.

The replace() function can be used to replace all occurrences of a character or a substring in a string and to return the new string. We can use replace() with an empty replacement to delete a substring. The also available delete() function is used to delete a range specified by two indices in place.

Additionally, a replaceWord() function exists, which does only replace whole words, i.e. words that are surrounded by word boundary characters like spaces, tabulators, or newlines.

The function multiReplace() is a variant, that can replace multiple substring/replacement pairs passed as tuples in one pass.

The function strip() can be used to remove multiple characters from a set at the start and the end of a string. The default character set is whitespace, and per default, strip() removes characters at both ends of the string. The function stripLineEnd() removes a single line end marker as \r, \n, \r\n, \f, \v from the end of the string, but only once.

Sometimes useful is the boolean function isEmptyOrWhitespace() which checks if a string is empty or contains only whitespace. Also useful can be the function repeat() which returns a string that contains the passed character or the passed string n times, and the function spaces() which returns a string containing only n spaces.

For single character tests, we have functions like isDigit(), isUpperAscii(), isLowerAscii(), isSpaceAscii, isAlphaNumeric(). Function isDigit() test for characters '0'..'9', isUpperAscii() for 'A'..'Z', islowerAscii() for 'a'..'z', isSpaceAsccii() for ASCII whitespace (' ', '\t') and isAlphaNumeric() test for lower or upper case ASCII letter or a decimal digit.

Function toLowerAscii() converts all the characters 'A'..'Z' to lower case, and toUpperAscii() converts all the characters 'a'..'z' to upper case. The function argument can be a string or just a single character. With capitalizeAscii(), we can convert the first ASCII character of a string to upper case.

The overloaded functions count() can be used to count the characters or substrings in a string, and countLines() is available to count the number of lines, where lines are separated by CR, LF, or CR-LF.

Sometimes, we may also need functions like formatFloat(), formatBiggestFloat(), or formatEng() to format float numbers for output purposes. You would have to consult the strutils API docs for all the format details. An intToStr() function with an argument to specify the minimal string length is also available. The string may get leading zeros for alignment.

Finally, some important functions are the parsing functions like parseFloat() or parseInt(), which convert strings to float or integer numbers. Both raise an exception when the string does not contain a valid number.

The strutils module contains some more not that often used functions, like functions to convert data to hexadecimal, octal, or binary representation, or to parse numbers back from that string representation into numbers. Other functions, like align(), center(), and indent() are available for string positioning. We will not try to describe these seldom-used functions here, as it is hard to remember the detailed behavior. You should skim the API docs and consult them when you need one of the exotic functions or when you have forgotten how to use a concrete function.[41]

Module parseutils

The module parseutils provides a set of functions for efficient and fast parsing of strings. The functions avoid the allocation of new strings by passing back results in var string parameters, and by returning the number of processed characters. The module parseutils is a good choice when we need efficient parsing of strings and the input strings have a simple structure. For more complicated input data, we may have to use RegEx or PEGs. Let us assume that we have a set of library names that include the version numbers, but we require the plain names. The function parseWhile() is a good candidate for this task:

import parseutils
var libs = ["libdconf.so.1.0.0", "libnice.so.10.11.0", "libwebkit2gtk-5.0.so.0.0.0"]
var l = newStringOfCap(128)
for s in libs:
  echo s.parseUntil(l, {'.', '-'})
  echo l
  echo s.parseWhile(l, {'a'..'z'})
  echo l

First, we allocate a string with enough capacity, so that the parse() functions can use it without having to do allocations. As we want to receive the plain names, using parseWhile() with a charset as the last parameter may be a possible solution. But as we see, this will not really work for webkit2gtk, which contains a digit in its name:

8
libdconf
8
libdconf
7
libnice
7
libnice
13
libwebkit2gtk
9
libwebkit

We can fix this by passing the extended char set {'a'..'z', '0'..'9'} to parseWhile() or by use of parseUntil() with a character set that does not belong to a name. Both functions return the number of processed characters and provide the captured string in the passed var parameter. Note that we can use the slice operator .. to specify character ranges for the charset parameter when the characters build a continuous sequence in the ASCII table.

A related function is skipUntil(), which we may use when we are more interested in the version numbers after the name:

  let p = s.skipUntil({'.', '-'})
  echo s[p .. ^1]

All these functions accept an optional start parameter as the last argument. A common use case is to use an integer position variable initialized with zero, which we increase by the returned value so that the parsing can continue at the current position in the string. The next example will use this strategy. For parseUntil() overloaded functions are available, which get not a charset, but a single character or a substring as a parameter. These functions stop parsing when the character or the substring is found and return that position.

Functions like parseInt() and parseFloat() can be used to extract numbers from strings:

import parseutils
var s = "In the year 2020 I gain 2.5 kg more fat."
var year: int
var value: float
var p = s.skipUntil({'0'..'9'})
p += parseUtils.parseInt(s, year, p)
p += s.skipUntil({'0'..'9'}, p)
p += parseUtils.parseFloat(s, value, p)
echo year, ": ", value # 2020: 2.5

In the above example, we used the module prefix, as strutils contains also a parseInt() and a parseFloat() function. The functions parseBin(), parseOct() and parseHex() behave similarly. Returned is the number of processed characters. We add the returned value to the start position so that parsing can continue at the new position.

There are some more functions available in this module, which we will not discuss further. It is enough that you know that this module exists and provides some efficient parsing functions. Whenever you should really need one of these procs, you would have to consult the API documentation for details.

Module strscans

The strscans module provides a scanf() macro, which can be used to extract substrings from textual user input. The content of the substrings is automatically converted to Nim variables of matching data types.

Processing of well-defined strings is easy, and with user-defined matcher functions, even text input with less strict shape can be processed.

Let us start with a simple example: We may have to create a program where the user should be able to create rectangles by entering the coordinates of two opposite corners in the form

Rect x1,y1,x2,y2
import std/strscans

var x1, y1, x2, y2: float
var name: string

let input = "Rect 10.0,20.0,100,200"

if scanf(input, "$w $f,$f,$f,$f", name, x1, y1, x2, y2):
  echo name, ' ', x1, ' ', y1, ' ', x2, ' ', y2 # Rect 10.0 20.0 100.0 200.0

The first parameter for the scanf() macro is the user input string, and the second parameter is a pattern string, that specifies how the input string should be processed. The following parameters are variables that get the results of the input evaluation. The second parameter has some similarities with a regular expression. The letters after the dollar sign specify the data type of the substrings, $i or $f ask to process an integer or a floating-point number, and $w requests to process an ASCII identifier. Other characters are captured verbatim, that is the space character after $w has to match a space in the input string, and the comma characters that separate the $f have to match commas in the input string. The scanf() macro supports capturing of some more data types, i.e. $c for an arbitrary character or $s for optional white space. The optional white space is not captured, just ignored. With the use of $s our program allows already a more flexible input string:

let input = "Rect10.0,    20.0,100,200"

if scanf(input, "$w$s$f,$s$f,$s$f,$s$f", name, x1, y1, x2, y2):
  echo name, ' ', x1, ' ', y1, ' ', x2, ' ', y2 # Rect 10.0 20.0 100.0 200.0

To allow processing even more flexible input strings, it is possible to use user-definable matchers in the form of Nim procs with a well-defined parameter signature. There are two different types of matcher procs supported — matchers to just skip a part of the input string, and capturing matchers. For the next example, we will use a proc which can skip various separators like commas, semicolons, or white space. And we will use a capturing matcher proc for the object name.

import std/strscans

proc sep(input: string; start: int; seps: set[char] = {' ',',',';'}): int =
  while start + result < input.len and input[start + result] in {' ','\t'}:
    inc(result)
  if start + result < input.len and input[start + result] in {';',','}:
    inc(result)
  while start + result < input.len and input[start + result] in {' ','\t'}:
    inc(result)

proc stt(input: string; strVal: var string; start: int; n: int): int =
  if input[start .. start + "Rect".high] == "Rect":
    strVal = "Rect"
    result = "Rect".len

var x1, y1, x2, y2: float
var name: string

let input = "Rect 10.0    ;20.0,100  ,  200"

if scanf(input, "${stt(0)}$s$f$[sep]$f$[sep]$f$[sep]$f", name, x1, y1, x2, y2):
  echo name, ' ', x1, ' ', y1, ' ', x2, ' ', y2 # Rect 10.0 20.0 100.0 200.0

The use of our user-definable matcher sep() allows separating the four numbers with a colon or a semicolon with arbitrary leading or trailing white space, or with only white space. Multiple colons or semicolons between two numbers resulting from a typo would be not permitted.

The signature for this matcher proc has this shape:

proc sep(input: string; start: int; seps: set[char] = {' ',',',';'}): int =

The first parameter is the string to process, and the second parameter is the position at which the processing should start. (In other words, the second parameter of integer type is the actual position in the input string, that position moves during the whole capering process from the start of the input string to its end. The end may not be reached if the capering fails at some point.) The last parameter has always the type set[char] with a default value indicating which characters that proc can process. Actually, a default value seems to be necessary, but the actual value seems not to matter. The proc returns the number of characters that should be skipped. Zero is a valid return value, so we can support optional separators. In most cases, separators are necessary to process the input string, but we can imagine input formats where separators are optional, e.g. when an integer number is followed by a name. A name never starts with a digit, so the boundary between the two values is well-defined. This none capturing matcher proc is called by use of $[sep] in the pattern string.

The signature of the capturing matcher proc has this shape:

proc stt(input: string; strVal: var string; start: int; n: int): int =

That proc also gets as parameters the input string and the start position, and has to return the number of processed characters. But additionally, a var parameter of arbitrary data type is used to return the result of the capture, and the last parameter with arbitrary data type can influence the capturing process. One possible use of the last parameter is to use an integer value to limit the maximum number of characters to process. This proc is called in the pattern string by using curly braces like ${stt(0)}.

Scanf() returns true, when all the parameters match, in that case, all the passed variables get assigned a value. Currently, scanf() does not support the capturing of optional data, as the whole processing stops, when one capture fails, i.e. when $i is used to request the capture of an integer value, but the input string does not contain decimal digits at the current capture position. In the same way, the whole capturing process stops when a user-defined capturing matcher returns zero, as no capturing is possible. So intermediate optional arguments are currently not supported. When the processing stops due to missing arguments, scanf() returns false, but the already processed captures still have a valid value assigned. In this way, we can use at least optional trailing arguments.

As the next example for the use of the scanf() macro, we will give a real-world example: A simple CAD (Computer-Aided Design) program has a PCB (Printed Circuit Board) mode, in which the user can create new PCB pads by entering the pad data in a text entry widget. A PCB pad is a rectangular-shaped copper field, which has an associated number and a name and maybe rounded corners. The user should be able to enter two 2D coordinates, the corner radius, an optional x/y translation for the next pad of the same size, followed by the number of pads to create, and the pad number and name. That is

pad x1 y1 x2 y2 r dx dy n num name

The first five arguments are mandatory, and the rest is optional, with default values. The user should be able to separate the arguments with white space, or with a colon or a semicolon. Additionally, the values x2 and y2 can be preceded with a + character to indicate that the x2, y2 tuple is not an absolute coordinate value, but the width and high of the pad.

A program fragment to process this form of user input may look like

import std/strscans

proc jecho(x: varargs[string, `$`]) =
  for el in x:
    stdout.write(el & " ")
  stdout.write('\n')
  stdout.flushfile

proc stt(input: string; strVal: var string; start: int; n: int): int =
  if input[start .. start + "pad".high] == "pad":
    strVal = "pad"
    result = "pad".len

proc pls(input: string; plusVal: var int; start: int; n: int): int =
  if input[start] == '+':
    plusVal = 1 # bool
    result = 1

proc sep(input: string; start: int; seps: set[char] = {' ',',',';'}): int =
  while start + result < input.len and input[start + result] in {' ','\t'}:
    inc(result)
  if start + result < input.len and input[start + result] in {';',','}:
    inc(result)
  while start + result < input.len and input[start + result] in {' ','\t'}:
    inc(result)

proc plus(input: string; plusVal: var int; start: int; n: int): int =
  result = sep(input, start)
  if input[start + result] == '+':
    plusVal = 1 # bool
    result += 1

var st: string
var x1, y1, x2, y2, <