jacob.eva/Reticulum
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Work on documentation

Browse Source
This commit is contained in:
Mark Qvist 2021-05-17 11:32:00 +02:00
parent 27736ee3f7
commit 4109cbc33d
2 changed files with 310 additions and 142 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
docs/source/examples.rst
View File

@ -10,56 +10,73 @@ You can use these examples to learn how to write your own programs.
Minimal Minimal
======= =======
This example can be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Minimal.py>`_.
The *Minimal* example demonstrates the bare-minimum setup required to connect to The *Minimal* example demonstrates the bare-minimum setup required to connect to
a Reticulum network from your program. In about five lines of code, you will a Reticulum network from your program. In about five lines of code, you will
have the Reticulum Network Stack initialised, and ready to pass traffic in your have the Reticulum Network Stack initialised, and ready to pass traffic in your
program. program.
.. literalinclude:: ../../Examples/Minimal.py
This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Minimal.py>`_.
.. _example-announce: .. _example-announce:
Announce Announce
======== ========
This example can be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Announce.py>`_.
The *Announce* example builds upon the previous example by exploring how to The *Announce* example builds upon the previous example by exploring how to
announce a destination on the network, and how to let your program receive announce a destination on the network, and how to let your program receive
notifications about announces from relevant destinations. notifications about announces from relevant destinations.
.. literalinclude:: ../../Examples/Announce.py
This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Announce.py>`_.
.. _example-broadcast: .. _example-broadcast:
Broadcast Broadcast
========= =========
This example can be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Broadcast.py>`_.
The *Broadcast* example explores how to transmit plaintext broadcast messages The *Broadcast* example explores how to transmit plaintext broadcast messages
over the network. over the network.
.. literalinclude:: ../../Examples/Broadcast.py
This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Broadcast.py>`_.
.. _example-echo: .. _example-echo:
Echo Echo
==== ====
This example can be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Echo.py>`_.
The *Echo* example demonstrates communication between two destinations using The *Echo* example demonstrates communication between two destinations using
the Packet interface. the Packet interface.
.. literalinclude:: ../../Examples/Echo.py
This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Echo.py>`_.
.. _example-link: .. _example-link:
Link Link
==== ====
This example can be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Link.py>`_.
The *Link* example explores establishing an encrypted link to a remote The *Link* example explores establishing an encrypted link to a remote
destination, and passing traffic back and forth over the link. destination, and passing traffic back and forth over the link.
.. literalinclude:: ../../Examples/Link.py
This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Link.py>`_.
.. _example-filetransfer: .. _example-filetransfer:
Filetransfer Filetransfer
============ ============
This example can be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Filetransfer.py>`_.
The *Filetransfer* example implements a basic file-server program that The *Filetransfer* example implements a basic file-server program that
allow clients to connect and download files. The program uses the Resource allow clients to connect and download files. The program uses the Resource
interface to efficiently pass files of any size over a Reticulum :ref:`Link<api-link>`. interface to efficiently pass files of any size over a Reticulum :ref:`Link<api-link>`.
.. literalinclude:: ../../Examples/Filetransfer.py
This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Filetransfer.py>`_.

419
docs/source/understanding.rst
View File

@ -10,13 +10,16 @@ develop networked applications using Reticulum.
This document is not an exhaustive source of information on Reticulum, at least not yet. Currently, This document is not an exhaustive source of information on Reticulum, at least not yet. Currently,
the best place to go for such information is the Python reference implementation of Reticulum, along the best place to go for such information is the Python reference implementation of Reticulum, along
with the API reference. with the code examples and API reference. It is however an essential resource to understanding the
general principles of Reticulum, how to apply them when creating your own networks or software.
After reading this document, you should be well-equipped to understand how a Reticulum network After reading this document, you should be well-equipped to understand how a Reticulum network
operates, what it can achieve, and how you can use it yourself. If you want to help out with the operates, what it can achieve, and how you can use it yourself. If you want to help out with the
development, this is also the place to start, since it will also provide a pretty clear overview of the development, this is also the place to start, since it will provide a pretty clear overview of the
sentiments and the philosophy behind Reticulum. sentiments and the philosophy behind Reticulum.
.. _understanding-motivation:
Motivation Motivation
========== ==========
@ -26,23 +29,25 @@ belief that it is highly desirable to create a cheap and reliable way to set up
communication network that can securely allow exchange of information between people and communication network that can securely allow exchange of information between people and
machines, with no central point of authority, control, censorship or barrier to entry. machines, with no central point of authority, control, censorship or barrier to entry.
Almost all of the various networking stacks in wide use today share a common limitation, namely Almost all of the various networking systems in use today share a common limitation, namely that they
that they require large amounts of coordination and trust to work. You can’t just plug in a bunch of require large amounts of coordination and trust to work, and to join the networks you need approval
ethernet cables to the same switch, or turn on a number of WiFi radios, and expect such a setup to of gatekeepers in control. This need for coordination and trust inevitably leads to an environment of
provide a reliable platform for communication. central control, where it's very easy for infrastructure operators or governments to control or alter
traffic, and censor or persecute unwanted actors.
This need for coordination and trust inevitably leads to an environment of control, where it's very
easy for infrastructure operators or governments to control or alter traffic.
Reticulum aims to require as little coordination and trust as possible. In fact, the only Reticulum aims to require as little coordination and trust as possible. In fact, the only
“coordination” required is to know how to get connected to a Reticulum network. Since Reticulum “coordination” required is to know the characteristics of physical medium carrying Reticulum traffic.
is medium agnostic, this could be whatever is best suited to the situation. In some cases, this might
be 1200 baud packet radio links over VHF frequencies, in other cases it might be a microwave Since Reticulum is completely medium agnostic, this could be whatever is best suited to the situation.
network using off-the-shelf radios. At the time of release of this document, the recommended setup In some cases, this might be 1200 baud packet radio links over VHF frequencies, in other cases it might
is using cheap LoRa radio modules with an open source firmware (see the chapter *Reference System be a microwave network using off-the-shelf radios. At the time of release of this document, the
Setup* ), connected to a small computer like a Raspberry Pi. As an example, the default reference recommended setup for development and testing is using LoRa radio modules with an open source firmware
setup provides a channel capacity of 5.4 Kbps, and a usable direct node-to-node range of around 15 (see the section :ref:`Reference System Setup<understanding-referencesystem>`), connected to a small
kilometers (indefinitely extendable by using multiple hops). computer like a Raspberry Pi. As an example, the default reference setup provides a channel capacity
of 5.4 Kbps, and a usable direct node-to-node range of around 15 kilometers (indefinitely extendable
by using multiple hops).
.. _understanding-goals:
Goals Goals
===== =====
@ -52,32 +57,33 @@ guide the design of Reticulum:
* **Fully useable as open source software stack** * **Fully useable as open source software stack**
Reticulum must be implemented, and be able to run using only open source software. This is Reticulum must be implemented with, and be able to run using only open source software. This is
critical to ensuring availability, security and transparency of the system. critical to ensuring the availability, security and transparency of the system.
* **Hardware layer agnosticism** * **Hardware layer agnosticism**
Reticulum shall be fully hardware agnostic, and should be useable over a wide range Reticulum shall be fully hardware agnostic, and shall be useable over a wide range
physical networking layers, such as data radios, serial lines, modems, handheld transceivers, physical networking layers, such as data radios, serial lines, modems, handheld transceivers,
wired ethernet, wifi, or anything else that can carry a digital data stream. Hardware made for wired ethernet, wifi, or anything else that can carry a digital data stream. Hardware made for
dedicated Reticulum use shall be as cheap as possible and use off-the-shelf components, so dedicated Reticulum use shall be as cheap as possible and use off-the-shelf components, so
it can be easily replicated. it can be easily replicated.
* **Very low bandwidth requirements** * **Very low bandwidth requirements**
Reticulum should be able to function reliably over links with a data capacity as low as *1,* Reticulum should be able to function reliably over links with a transmission capacity as low
*bps*. as *1,000 bps*.
* **Encryption by default** * **Encryption by default**
Reticulum must use encryption by default where possible and applicable. Reticulum must use encryption by default where possible and applicable.
* **Unlicensed use** * **Unlicensed use**
Reticulum shall be functional over physical communication mediums that do not require any Reticulum shall be functional over physical communication mediums that do not require any
form of license to use. Reticulum must be designed in a way, so it is usable over ISM radio form of license to use. Reticulum must be designed in a way, so it is usable over ISM radio
frequency bands, and can provide functional long distance links in such conditions. frequency bands, and can provide functional long distance links in such conditions, for example
by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules.
* **Supplied software** * **Supplied software**
Apart from the core networking stack and API, that allows any developer to build Apart from the core networking stack and API, that allows a developer to build
applications with Reticulum, a basic communication suite using Reticulum must be applications with Reticulum, a basic communication suite using Reticulum must be
implemented and released at the same time as Reticulum itself. This shall serve both as a implemented and released at the same time as Reticulum itself. This shall serve both as a
functional communication suite, and as an example and learning resource to others wishing functional communication suite, and as an example and learning resource to others wishing
to build applications with Reticulum. to build applications with Reticulum.
* **Ease of use** * **Ease of use**
The reference implementation of Reticulum is written in Python, to make it very easy to use The reference implementation of Reticulum is written in Python, to make it easy to use
and understand. Any programmer with only basic experience should be able to use and understand. A programmer with only basic experience should be able to use
Reticulum in their own applications. Reticulum in their own applications.
* **Low cost** * **Low cost**
It shall be as cheap as possible to deploy a communication system based on Reticulum. This It shall be as cheap as possible to deploy a communication system based on Reticulum. This
@ -85,30 +91,42 @@ guide the design of Reticulum:
own. The cost of setting up a functioning node should be less than $100 even if all parts own. The cost of setting up a functioning node should be less than $100 even if all parts
needs to be purchased. needs to be purchased.
.. _understanding-basicfunctionality:
Introduction & Basic Functionality Introduction & Basic Functionality
================================== ==================================
Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at it’s Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at it’s
core *message oriented* , but can provide connection oriented sessions. It is suited for both local core a *message oriented* system. It is suited for both local point-to-point or point-to-multipoint
point-to-point or point-to-multipoint scenarios where alle nodes are within range of each other, as scenarios where alle nodes are within range of each other, as well as scenarios where packets need
well as scenarios where packets need to be transported over multiple hops to reach the recipient. to be transported over multiple hops to reach the recipient.
Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead
Reticulum uses the singular concept of *destinations*. Any application using Reticulum as it’s Reticulum uses the singular concept of *destinations*. Any application using Reticulum as it’s
networking stack will need to create one or more destinations to receive data, and know the networking stack will need to create one or more destinations to receive data, and know the
destinations it needs to send data to. destinations it needs to send data to.
Reticulum encrypts all data by default using public-key cryptography. Any message sent to a All destinations in Reticulum are represented internally as 10 bytes, derived from truncating a full
destination is encrypted with that destinations public key. Reticulum also offers symmetric key SHA-256 hash of identifying characteristics of the destination. To users, the destination addresses
encryption for group-oriented communications, as well as unencrypted packets for broadcast will be displayed as 10 bytes in hexadecimal representation, as in the following example: ``<80e29bf7cccaf31431b3>``.
purposes, or situations where you need the communication to be in plain text. The multi-hop
transport, coordination, verification and reliability layers are fully autonomous and based on public By default Reticulum encrypts all data using public-key cryptography. Any message sent to a
key cryptography. destination is encrypted with that destinations public key. Reticulum can also set up an encrypted
channel to a destination with *Perfect Forward Secrecy* and *Initiator Anonymity* using a elliptic
curve cryptography and ephemeral keys derived from a Diffie Hellman exchange on the SECP256R1 curve.
In Reticulum terminology, this is called a *Link*.
Reticulum also offers symmetric key encryption for group-oriented communications, as well as
unencrypted packets for broadcast purposes, or situations where you need the communication to be in
plain text. The multi-hop transport, coordination, verification and reliability layers are fully
autonomous and based on public key cryptography.
Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports, Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports,
and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or
private IP networks. private IP networks.
.. _understanding-destinations:
Destinations Destinations
------------ ------------
@ -127,57 +145,80 @@ destinations. Reticulum uses three different basic destination types, and one sp
can by many. can by many.
* **Plain** * **Plain**
A *plain* destination type is unencrypted, and suited for traffic that should be broadcast to a A *plain* destination type is unencrypted, and suited for traffic that should be broadcast to a
number of users, or should be readable by anyone. number of users, or should be readable by anyone. Traffic to a *plain* destination is not encrypted.
* **Link** * **Link**
A *link* is a special destination type, that serves as an abstract channel between two *single* A *link* is a special destination type, that serves as an abstract channel to a *single*
destinations, directly connected or over multiple hops. The *link* also offers reliability and destination, directly connected or over multiple hops. The *link* also offers reliability and
more efficient encryption, and as such is useful even when nodes are directly connected. more efficient encryption, forward secrecy, initiator anonymity, and as such can be useful even
when a node is directly reachable.
.. _understanding-destinationnaming:
Destination Naming Destination Naming
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
Destinations are created and named in an easy to understand dotted notation of *aspects* , and Destinations are created and named in an easy to understand dotted notation of *aspects* , and
represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The
top level aspect should always be the a unique identifier for the application using the destination. top level aspect should always be a unique identifier for the application using the destination.
The next levels of aspects can be defined in any way by the creator of the application. For example, The next levels of aspects can be defined in any way by the creator of the application. For example,
a destination for a messaging application could be made up of the application name and a username, a destination for a environmental monitoring application could be made up of the application name, a
and look like this: device type and measurement type, like this:
.. code-block:: .. code-block:: text
name: simplemessenger.someuser hash: 2a7ddfab5213f916dea app name : environmentlogger
aspects : remotesensor, temperature
full name : environmentlogger.remotesensor.temperature
hash : fa7ddfab5213f916dea
For the *single* destination, Reticulum will automatically append the associated public key as a For the *single* destination, Reticulum will automatically append the associated public key as a
destination aspect before hashing. This is done to ensure only the correct destination is reached, destination aspect before hashing. This is done to ensure only the correct destination is reached,
since anyone can listen to any destination name. Appending the public key ensures that a given since anyone can listen to any destination name. Appending the public key ensures that a given
packet is only directed at the destination that holds the corresponding private key to decrypt the packet is only directed at the destination that holds the corresponding private key to decrypt the
packet. It is important to understand that anyone can use the destination name packet.
*simplemessenger.myusername* , but each person that does so will still have a different destination
hash, because their public keys will differ. In actual use of *single* destination naming, it is advisable
not to use any uniquely identifying features in aspect naming, though. In the simple messenger
example, when using *single* destinations, we would instead use a destination naming scheme such
as *simplemessenger.user* where appending the public key expands the destination into a uniquely
identifying one.
To recap, the destination types should be used in the following situations: **Take note!** There is a very important concept to understand here:
* Anyone can use the destination name ``environmentlogger.remotesensor.temperature``
* Each destination that does so will still have a unique destination hash, and thus be uniquely
addressable, because their public keys will differ.
In actual use of *single* destination naming, it is advisable not to use any uniquely identifying
features in aspect naming. Aspect names should be general terms describing what kind of destination
is represented. The uniquely identifying aspect is always acheived by the appending the public key,
which expands the destination into a uniquely identifyable one.
Any destination on a Reticulum network can be addressed and reached simply by knowning its
destination hash (and public key, but if the public key is not known, it can be requested from the
network simply by knowing the destination hash). The use of app names and aspects makes it easy to
structure Reticulum programs and makes it possible to filter what information and data your program
receives.
To recap, the different destination types should be used in the following situations:
* **Single** * **Single**
When private communication between two endpoints is needed. Supports routing. When private communication between two endpoints is needed. Supports multiple hops.
* **Group** * **Group**
When private communication between two or more endpoints is needed. More efficient in When private communication between two or more endpoints is needed. More efficient in
data usage than *single* destinations. Supports routing indirectly, but must first be established data usage than *single* destinations. Supports multiple hops indirectly, but must first be
through a *single* destination. established through a *single* destination.
* **Plain** * **Plain**
When plain-text communication is desirable, for example when broadcasting information. When plain-text communication is desirable, for example when broadcasting information.
To communicate with a *single* destination, you need to know it’s public key. Any method for To communicate with a *single* destination, you need to know it’s public key. Any method for
obtaining the public key is valid, but Reticulum includes a simple mechanism for making other obtaining the public key is valid, but Reticulum includes a simple mechanism for making other
nodes aware of your destinations public key, called the *announce*. nodes aware of your destinations public key, called the *announce*. It is also possible to request
an unknown public key from the network, as all participating nodes serve as a distributed ledger
of public keys.
Note that this information could be shared and verified in many other ways, and that it is therefore Note that public key information can be shared and verified in many other ways than using the
not required to use the announce functionality, although it is by far the easiest, and should probably built-in methodology, and that it is therefore not required to use the announce/request functionality.
be used if you are not confident in how to verify public keys and signatures manually. It is by far the easiest though, and should definitely be used if there is not a good reason for
doing it differently.
.. _understanding-keyannouncements:
Public Key Announcements Public Key Announcements
------------------------ ------------------------
@ -204,8 +245,11 @@ will be implicit in almost all cases. If a destination name is not entirely impl
included in the application specific data part that will allow the receiver to infer the naming. included in the application specific data part that will allow the receiver to infer the naming.
It is important to note that announcements will be forwarded throughout the network according to a It is important to note that announcements will be forwarded throughout the network according to a
certain pattern. This will be detailed later. Seeing how *single* destinations are always tied to a certain pattern. This will be detailed later.
private/public key pair leads us to the next topic.
Seeing how *single* destinations are always tied to a private/public key pair leads us to the next topic.
.. _understanding-identities:
Identities Identities
---------- ----------
@ -227,51 +271,65 @@ application. Destinations created will then be linked to this identity to allow
reach the user. In such a case it is of great importance to store the user’s identity securely and reach the user. In such a case it is of great importance to store the user’s identity securely and
privately. privately.
.. _understanding-gettingfurther:
Getting Further Getting Further
--------------- ---------------
The above functions and principles form the core of Reticulum, and would suffice to create The above functions and principles form the core of Reticulum, and would suffice to create
functional networked applications in local clusters, for example over radio links where all interested functional networked applications in local clusters, for example over radio links where all interested
nodes can hear each other. But to be truly useful, we need a way to go further. In the next chapter, nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple
two concepts that allow this will be introduced, *paths* and *resources*. hops in the network. In the next sections, two concepts that allow this will be introduced, *paths* and
*links*.
.. _understanding-transport:
Reticulum Transport Reticulum Transport
=================== ===================
I have purposefully avoided the term routing until now, and will continue to do so, because the The term routing has been purposefully avoided until now. The current methods of routing used in IP-based
current methods of routing used in IP based networks are fundamentally incompatible for the link networks are fundamentally incompatible with the physical link types that Reticulum was designed to handle.
types that Reticulum was designed to handle. These routing methodologies assume trust at the These routing methodologies assume trust at the physical layer, and often needs a lot more bandwidth than
physical layer. Since Reticulum is designed to run over open radio spectrum, no such trust exists. Reticulum can assume is available.
Furthermore, existing routing protocols like BGP or OSPF carry too much overhead to be
practically useable over bandwidth-limited, high-latency links. Since Reticulum is designed to run over open radio spectrum, no such trust exists, and bandwidth is often
very limited. Existing routing protocols like BGP or OSPF carry too much overhead to be practically
useable over bandwidth-limited, high-latency links.
To overcome such challenges, Reticulum’s *Transport* system uses public-key cryptography to To overcome such challenges, Reticulum’s *Transport* system uses public-key cryptography to
implement the concept of *paths* that allow discovery of how to get information to a certain implement the concept of *paths* that allow discovery of how to get information to a certain
destination, and *resources* that help alleviate congestion and make reliable communication more destination, and *resources* that help make reliable data transfer more efficient.
efficient and less bandwidth-hungry.
Threading a Path .. _understanding-paths:
----------------
Reaching the Destination
------------------------
In networks with changing topology and trustless connectivity, nodes need a way to establish In networks with changing topology and trustless connectivity, nodes need a way to establish
*verified connectivity* with each other. To do this, the following process is employed: *verified connectivity* with each other. Since the network is assumed to be trustless, Reticulum
must provide a way to guarantee that the peer you are communicating with is actually who you
expect. To do this, the following process is employed:
* First, the node that wishes to establish connectivity will send out a special packet, that * | First, the node that wishes to establish connectivity will send out a special packet, that
traverses the network and locates the desired destination. Along the way, the nodes that traverses the network and locates the desired destination. Along the way, the nodes that
forward the packet will take note of this *link request*. forward the packet will take note of this *link request*.
* Second, if the destination accepts the *link request* , it will send back a packet that proves the
* | Second, if the destination accepts the *link request* , it will send back a packet that proves the
authenticity of it’s identity (and the receipt of the link request) to the initiating node. All authenticity of it’s identity (and the receipt of the link request) to the initiating node. All
nodes that initially forwarded the packet will also be able to verify this proof, and thus nodes that initially forwarded the packet will also be able to verify this proof, and thus
accept the validity of the *link* throughout the network. accept the validity of the *link* throughout the network.
* When the validity of the *link* has been accepted by forwarding nodes, these nodes will
* | When the validity of the *link* has been accepted by forwarding nodes, these nodes will
remember the *link* , and it can subsequently be used by referring to a hash representing it. remember the *link* , and it can subsequently be used by referring to a hash representing it.
* As a part of the *link request* , a Diffie-Hellman key exchange takes place, that sets up an
* | As a part of the *link request* , a Diffie-Hellman key exchange takes place, that sets up an
efficient symmetrically encrypted tunnel between the two nodes, using elliptic curve efficient symmetrically encrypted tunnel between the two nodes, using elliptic curve
cryptography. As such, this mode of communication is preferred, even for situations when cryptography. As such, this mode of communication is preferred, even for situations when
nodes can directly communicate, when the amount of data to be exchanged numbers in the nodes can directly communicate, when the amount of data to be exchanged numbers in the
tens of packets. tens of packets.
* When a *link* has been set up, it automatically provides message receipt functionality, so the
* | When a *link* has been set up, it automatically provides message receipt functionality, so the
sending node can obtain verified confirmation that the information reached the intended sending node can obtain verified confirmation that the information reached the intended
recipient. recipient.
@ -280,37 +338,44 @@ recap what purposes this serves. We first ensure that the node answering our req
one we want to communicate with, and not a malicious actor pretending to be so. At the same time one we want to communicate with, and not a malicious actor pretending to be so. At the same time
we establish an efficient encrypted channel. The setup of this is relatively cheap in terms of we establish an efficient encrypted channel. The setup of this is relatively cheap in terms of
bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will also bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will also
rotate encryption keys, but the link can also be kept alive for longer periods of time, if this is
more suitable to the application. The amount of bandwidth used on keeping a link open is practically
negligible. The procedure also inserts the *link id* , a hash calculated from the link request packet,
into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each
other simply by referring to this *link id*.
rotate encryption keys (keys can also be rotated over an existing path), but the link can also be kept Step 1: Pathfinding
alive for longer periods of time, if this is more suitable to the application. The amount of bandwidth ^^^^^^^^^^^^^^^^^^^
used on keeping a link open is practically negligible. The procedure also inserts the *link id* , a hash
calculated from the link request packet, into the memory of forwarding nodes, which means that the
communicating nodes can thereafter reach each other simply by referring to this *link id*.
**Step 1, pathfinding**
The pathfinding method builds on the *announce* functionality discussed earlier. When an announce The pathfinding method builds on the *announce* functionality discussed earlier. When an announce
is sent out by a node, it will be forwarded by any node receiving it, but according to some specific is sent out by a node, it will be forwarded by any node receiving it, but according to some specific
rules: rules:
* If this announce has already been received before, ignore it. * | If this announce has already been received before, ignore it.
* Record into a table which node the announce was received from, and how many times in
* | Record into a table which node the announce was received from, and how many times in
total it has been retransmitted to get here. total it has been retransmitted to get here.
* If the announce has been retransmitted *m+1* times, it will not be forwarded. By default, *m* is
* | If the announce has been retransmitted *m+1* times, it will not be forwarded. By default, *m* is
set to 18. set to 18.
* The announce will be assigned a delay *d* = *ch* seconds, where *c* is a decay constant, by
* | The announce will be assigned a delay *d* = c\ :sup:`h` seconds, where *c* is a decay constant, by
default 2, and *h* is the amount of times this packet has already been forwarded. default 2, and *h* is the amount of times this packet has already been forwarded.
* The packet will be given a priority *p = 1/d*.
* If at least *d* seconds has passed since the announce was received, and no other packets with a * | The packet will be given a priority *p = 1/d*.
* | If at least *d* seconds has passed since the announce was received, and no other packets with a
priority higher than *p* are waiting in the queue (see Packet Prioritisation), and the channel is priority higher than *p* are waiting in the queue (see Packet Prioritisation), and the channel is
not utilized by other traffic, the announce will be forwarded. not utilized by other traffic, the announce will be forwarded.
* If no other nodes are heard retransmitting the announce with a greater hop count than when
* | If no other nodes are heard retransmitting the announce with a greater hop count than when
it left this node, transmitting it will be retried *r* times. By default, *r* is set to 2. Retries follow it left this node, transmitting it will be retried *r* times. By default, *r* is set to 2. Retries follow
same rules as above, with the exception that it must wait for at least *d = ch+1 + t* seconds, ie., same rules as above, with the exception that it must wait for at least *d* = c\ :sup:`h+1` + t seconds, ie.,
the amount of time it would take the next node to retransmit the packet. By default, *t* is set to the amount of time it would take the next node to retransmit the packet. By default, *t* is set to
10. 10.
* If a newer announce from the same destination arrives, while an identical one is already in
* | If a newer announce from the same destination arrives, while an identical one is already in
the queue, the newest announce is discarded. If the newest announce contains different the queue, the newest announce is discarded. If the newest announce contains different
application specific data, it will replace the old announce, but will use *d* and *p* of the old application specific data, it will replace the old announce, but will use *d* and *p* of the old
announce. announce.
@ -319,17 +384,16 @@ Once an announce has reached a node in the network, any other node in direct con
node will be able to reach the destination the announce originated from, simply by sending a packet node will be able to reach the destination the announce originated from, simply by sending a packet
addressed to that destination. Any node with knowledge of the announce will be able to direct the addressed to that destination. Any node with knowledge of the announce will be able to direct the
packet towards the destination by looking up the next node with the shortest amount of hops to the packet towards the destination by looking up the next node with the shortest amount of hops to the
destination. The specifics of this process is detailed in *Path Calculation*. destination.
According to these rules and default constants, an announce will propagate throughout the network According to these rules and default constants, an announce will propagate throughout the network
in a predictable way. In an example network utilising the default constants, and with an average link in a predictable way. In an example network utilising the default constants, and with an average link
distance of *Lavg =* 15 kilometers, an announce will be able to propagate outwards to a radius of 180 distance of *Lavg =* 15 kilometers, an announce will be able to propagate outwards to a radius of 180
kilometers in 34 minutes, and a *maximum announce radius* of 270 kilometers in approximately 3 kilometers in 34 minutes, and a *maximum announce radius* of 270 kilometers in approximately 3
days. Methods for overcoming the distance limitation of *m * Lavg* will be introduced later in this days.
chapter.
**Step 2, link establishment** Step 2: Link Establishment
^^^^^^^^^^^^^^^^^^^^^^^^^^
After seeing how the conditions for finding a path through the network are created, we will now After seeing how the conditions for finding a path through the network are created, we will now
explore how two nodes can establish reliable communications over multiple hops. The *link* in explore how two nodes can establish reliable communications over multiple hops. The *link* in
@ -338,25 +402,30 @@ as an abstract channel, that can be open for any amount of time, and can span an
of hops, where information will be exchanged between two nodes. of hops, where information will be exchanged between two nodes.
* When a node in the network wants to establish verified connectivity with another node, it * | When a node in the network wants to establish verified connectivity with another node, it
will create a *link request* packet, and broadcast it. will create a *link request* packet, and broadcast it.
* The *link request* packet contains the destination hash *Hd* , and an asymmetrically encrypted
* | The *link request* packet contains the destination hash *Hd* , and an asymmetrically encrypted
part containing the following data: The source hash *Hs* , a symmetric key *Lk* , a truncated part containing the following data: The source hash *Hs* , a symmetric key *Lk* , a truncated
hash of a random number *Hr* , and a signature *S* of the plaintext values of *Hd* , *Hs* , *Lk* and *Hr*. hash of a random number *Hr* , and a signature *S* of the plaintext values of *Hd* , *Hs* , *Lk* and *Hr*.
* The broadcasted packet will be directed through the network according to the rules laid out
* | The broadcasted packet will be directed through the network according to the rules laid out
previously. previously.
* Any node that forwards the link request will store a *link id* in it’s *link table* , along with the
* | Any node that forwards the link request will store a *link id* in it’s *link table* , along with the
amount of hops the packet had taken when received. The link id is a hash of the entire link amount of hops the packet had taken when received. The link id is a hash of the entire link
request packet. If the path is not *proven* within some set amount of time, the entry will be request packet. If the path is not *proven* within some set amount of time, the entry will be
dropped from the table again. dropped from the table again.
* When the destination receives the link request packet, it will decide whether to accept the
* | When the destination receives the link request packet, it will decide whether to accept the
request. If it is accepted, it will create a special packet called a *proof*. A *proof* is a simple request. If it is accepted, it will create a special packet called a *proof*. A *proof* is a simple
construct, consisting of a truncated hash of the message that needs to be proven, and a construct, consisting of a truncated hash of the message that needs to be proven, and a
signature (made by the destination’s private key) of this hash. This *proof* effectively verifies signature (made by the destination’s private key) of this hash. This *proof* effectively verifies
that the intended recipient got the packet, and also serves to verify the discovered path that the intended recipient got the packet, and also serves to verify the discovered path
through the network. Since the *proof* hash matches the *path id* in the intermediary nodes’ through the network. Since the *proof* hash matches the *path id* in the intermediary nodes’
*path tables* , the intermediary nodes can forward the proof all the way back to the source. *path tables* , the intermediary nodes can forward the proof all the way back to the source.
* When the source receives the *proof* , it will know unequivocally that a verified path has been
* | When the source receives the *proof* , it will know unequivocally that a verified path has been
established to the destination, and that information can now be exchanged reliably and established to the destination, and that information can now be exchanged reliably and
securely. securely.
@ -372,17 +441,12 @@ of Reticulum, such a retransmission does not need to travel the entire length of
If a packet is lost on the 8th hop of a 12 hop path, it can be fetched from the last hop that received it If a packet is lost on the 8th hop of a 12 hop path, it can be fetched from the last hop that received it
reliably. reliably.
Crossing Continents .. _understanding-resources:
-------------------
When a packet needs to travel farther than local network topology knowledge stretches, a system of Resources
geographical or topological hinting is used to direct the packet towards a network segment with ---------
direct knowledge of the intended destination. This functionality is currently left out of the protocol
for simplicity of testing other parts, but will be activated in a future release. For more information
on when, refer to the roadmap on the website.
Resourceful Memory TODO: Write
------------------
In traditional networks, large amounts of data is rapidly exchanged with very low latency. Links of In traditional networks, large amounts of data is rapidly exchanged with very low latency. Links of
several thousand kilometers will often only have round-trip latency in the tens of milliseconds, and several thousand kilometers will often only have round-trip latency in the tens of milliseconds, and
@ -415,6 +479,8 @@ certain destination, and as such the network as a whole operates as a distribute
For more details on how the caching works and is used, see the reference implementation source For more details on how the caching works and is used, see the reference implementation source
code. code.
.. _understanding-referencesystem:
Reference System Setup Reference System Setup
====================== ======================
@ -450,10 +516,9 @@ into the future. The current Reference System Setup is as follows:
* **Channel Access Device** * **Channel Access Device**
A data radio consisting of a LoRa radio module, and a microcontroller with open source A data radio consisting of a LoRa radio module, and a microcontroller with open source
firmware, that can connect to host devices via USB. It operates in either the 430, 868 or 900 firmware, that can connect to host devices via USB. It operates in either the 430, 868 or 900
MHz frequency bands. More details on the exact parts and how to get/make one can be MHz frequency bands. More details can be found on the `RNode Page <https://unsigned.io/rnode>`_.
found on the website.
* **Host device** * **Host device**
Any computer device running Linux and Python. A Raspberry Pi with Raspbian is Any computer device running Linux and Python. A Raspberry Pi with a Debian based OS is
recommended. recommended.
* **Software stack** * **Software stack**
The current Reference Implementation Release of Reticulum, running on a Debian based The current Reference Implementation Release of Reticulum, running on a Debian based
@ -461,11 +526,13 @@ into the future. The current Reference System Setup is as follows:
It is very important to note, that the reference channel access device **does not** use the LoRaWAN It is very important to note, that the reference channel access device **does not** use the LoRaWAN
standard, but uses a custom MAC layer on top of the plain LoRa modulation! As such, you will standard, but uses a custom MAC layer on top of the plain LoRa modulation! As such, you will
need a plain LoRa radio module connected to an MCU with the correct Reticulum firmware. Full need a plain LoRa radio module connected to an MCU with the correct firmware. Full details on how to
details on how to get or make such a device is available on the website. get or make such a device is available on the `RNode Page <https://unsigned.io/rnode>`_.
With the current reference setup, it should be possible to get on a Reticulum network for around 70$ With the current reference setup, it should be possible to get on a Reticulum network for around 100$
even if you have none of the hardware already. even if you have none of the hardware already, and need to purchase everything.
.. _understanding-protocolspecifics:
Protocol Specifics Protocol Specifics
================== ==================
@ -474,30 +541,114 @@ This chapter will detail protocol specific information that is essential to the
Reticulum, but non critical in understanding how the protocol works on a general level. It should be Reticulum, but non critical in understanding how the protocol works on a general level. It should be
treated more as a reference than as essential reading. treated more as a reference than as essential reading.
Node Types Node Types
---------- ----------
Currently Reticulum defines two node types, the *Station* and the *Peer*. A node is a *station* if it fixed Currently Reticulum defines two node types, the *Station* and the *Peer*. A node is a *station* if it fixed
in one place, and if it is intended to be kept online at all times. Otherwise the node is a *peer*. This in one place, and if it is intended to be kept online most of the time. Otherwise the node is a *peer*.
distinction is made by the user configuring the node, and is used to determine what nodes on the This distinction is made by the user configuring the node, and is used to determine what nodes on the
network will help forward traffic, and what nodes rely on other nodes for connectivity. network will help forward traffic, and what nodes rely on other nodes for connectivity.
If a node is a *Peer* it should be given the configuration directive ``enable_transport = No``.
If it is a *Station*, it should be given the configuration directive ``enable_transport = Yes``.
Packet Prioritisation Packet Prioritisation
--------------------- ---------------------
*The packet prioritisation algorithms are subject to rapid change at the moment, and for now, they Currently, Reticulum is completely priority-agnostic regarding general traffic. All traffic is handled
are not documented here. See the reference implementation for more info on how this functionality on a first-come, first-serve basis. Announce re-transmission are handled according to the re-transmission
works.* times and priorities described earlier in this chapter.
Path Calculation It is possible that a prioritisation engine could be added to Reticulum in the future, but in
---------------- the light of Reticulums goal of equal access, doing so would need to be the subject of careful
investigation of the consequences first.
*The path calculation algorithms are subject to rapid change at the moment, and for now, they are
not documented here. See the reference implementation for more info on how this functionality
works.*
Binary Packet Format Binary Packet Format
-------------------- --------------------
*The binary packet format is subject to rapid change at the moment, and for now, it is not .. code-block:: text
documented here. See the reference implementation for the specific details on this topic.*
== Reticulum Wire Format ======
A Reticulum packet is composed of the following fields:
[HEADER 2 bytes] [ADDRESSES 10/20 bytes] [CONTEXT 1 byte] [DATA 0-477 bytes]
* The HEADER field is 2 bytes long.
* Byte 1: [Header Type], [Propagation Type], [Destination Type] and [Packet Type]
* Byte 2: Number of hops
* The ADDRESSES field contains either 1 or 2 addresses.
* Each address is 10 bytes long.
* The Header Type flag in the HEADER field determines
whether the ADDRESSES field contains 1 or 2 addresses.
* Addresses are Reticulum hashes truncated to 10 bytes.
* The CONTEXT field is 1 byte.
* It is used by Reticulum to determine packet context.
* The DATA field is between 0 and 477 bytes.
* It contains the packets data payload.
Header Types
-----------------
type 1 00 Two byte header, one 10 byte address field
type 2 01 Two byte header, two 10 byte address fields
type 3 10 Reserved
type 4 11 Reserved
Propagation Types
-----------------
broadcast 00
transport 01
reserved 10
reserved 11
Destination Types
-----------------
single 00
group 01
plain 10
link 11
Packet Types
-----------------
data 00
announce 01
link request 10
proof 11
+- Packet Example -+
HEADER FIELD ADDRESSES FIELD CONTEXT FIELD DATA FIELD
_______|_______ ________________|________________ ________|______ __|_
| | | | | | | |
01010000 00000100 [ADDR1, 10 bytes] [ADDR2, 10 bytes] [CONTEXT, 1 byte] [DATA]
| | | | |
| | | | +-- Hops = 4
| | | +------- Packet Type = DATA
| | +--------- Destination Type = SINGLE
| +----------- Propagation Type = TRANSPORT
+------------- Header Type = HEADER_2 (two byte header, two address fields)
+- Packet Example -+
HEADER FIELD ADDRESSES FIELD CONTEXT FIELD DATA FIELD
_______|_______ _______|_______ ________|______ __|_
| | | | | | | |
00000000 00000111 [ADDR1, 10 bytes] [CONTEXT, 1 byte] [DATA]
| | | | |
| | | | +-- Hops = 7
| | | +------- Packet Type = DATA
| | +--------- Destination Type = SINGLE
| +----------- Propagation Type = BROADCAST
+------------- Header Type = HEADER_1 (two byte header, one address field)