The twister P2P Book

About this book

Community-driven guide to twister p2p, based on Wiki data from various sources, primarily from the community repositories.

It uses the mdBook builder to create a static HTML book,
which is also available for online reading at the community website.

To download the latest book sources or report a bug, visit project repository!

About twister p2p

The twister (in lowercase) is an open-source, peer-to-peer microblogging platform released by Miguel Freitas in 2013.

According to the announcement, on June 25, 2020, official project maintenance was discontinued, but the network remains active as it does not require centralized servers. Since 2021, development has been maintained by the community.

About twisterarmy

The twisterarmy is a non-profit organization created in 2021 by a few users with the intention of supporting the original twister p2p network in the community format.

Community resources include the latest editions of twister-core, twister-html, and other projects also created by various individuals, such as related software collections, analytics tools, cloud servers, and more.

The organization's goal is to ensure that the source code is buildable and the network is connectable.

Feel free to open an issue or mention @twisterarmy if you find that the information provided is incorrect, contains mistakes, or if you have any questions!

Versioning

This book follows the Semantic Versioning 2.0.0 structure for stable releases:

• MAJOR - the book edition
• MINOR - new articles, URI changes
• PATCH - existing content updates, typo corrections, etc.

Online

Latest online version is available on the community website. This version is updated manually from the upstream repository to test future stable releases. It also includes the content generation timestamp in the page footer.

Download

To obtain the latest book's sources (/src), which also include a compiled HTML version for offline reading (/book), visit project repository.

For previous stable snaps, refer to the Releases page.

Mirrors

Internet

• https://twisterarmy.github.io/book/ - the official online version with manual updates from the upstream repository, operated by twisterarmy

Yggdrasil

• http://[300:17a8:aabf:108f::33] - an online version with daily auto-updates from the upstream repository, operated by YGGverse

Dependencies

• rustc, cargo - see rustup
  • cargo install mdbook - mdBook builder CLI
  • cargo install mdbook-alerts - preprocessor to add GitHub Flavored Markdown's Alerts
• python3 - to run the preprocessor.py on build

Get this builder

• git clone https://github.com/twisterarmy/book.git && cd book
• chmod +x build.sh

Get latest book sources

./build.sh

Launch

Compiled HTML version should be available in the /book directory.

To read in the browser using a local server, run:

mdbook serve --open

Connect new project

If you have a new Wiki (or documentation) repository for your project to connect, please define it in the Book's webroot location, e.g:

• /twister-core
• /twister-html
• /your-app

Formatting

Please follow these recommendations when creating or editing content:

• exclude remote content dependencies, such as status badges, images, and other media that require a remote connection to function; if the media file is required, it should be associated with your upstream repository for cloning;
• prefer absolute URLs for local referencing, as the book route may be different;
• this book supports GitHub Flavored Markdown's Alerts - feel free to decorate the content semantically!

Edit existing sources

This project uses build.sh to aggregate the book data from different sources.
If you want to make any changes, please submit your edits to the relevant Wiki providers:

• Summary
  • About
• twister-core
  • miguelfreitas
  • twisterarmy
• Mining tools
  • twisterad
  • CudaMiner-twister
• Network tools
  • twister-seeder
• Bridge tools
  • twister-rss-bot

Feel free to open an Issue if you have any questions!

Run the mirror

Make the book accessible to others by setting up an additional repository or web mirror for online reading!

Repository mirror

Some self-hosted repository providers, such as Gitea, allow you to easily clone the upstream repository using web UI and create a mirror out of the box. Follow the software documentation for details!

Alternatively, you can set up your own public (or private) repository by running:

git clone https://github.com/twisterarmy/book.git

and then set up the auto-updates using crontab and git pull command (example).

Web mirror for online reading

The upstream repository contains a precompiled HTML version in the /book directory, which you can set up as the web root in your nginx or Apache web server.

Update the mirror with crontab

For daily auto-updates, consider using a crontab task with the git pull command running from the cloned book project:

@daily cd /var/www/book && /usr/bin/git pull

• if the repository is located at /var/www/book and its HTML version is located in the original book directory
• for more crontab examples, visit this beautiful guide

Announce new mirror

If you're the public mirror maintainer, feel free to share your link with others on the Releases page in the Mirrors section!

Thanks to everyone who made the content for this book;
who uses twister and helps to run its infrastructure, especially:

• @myleneb - for the community logo based on @theme_nin and for the inspiration to work on the twisterarmy project
• @denis - for assisting with the setup of twister-seeder, whose experience was used to create the guide section
• @vegos - for mining the blocks all this time
• @twisterp2p - for participating the public peering
• @mfreitas - for creating the twister P2P network!

About twister-core

twister-core is the primary twister networking application, released by Miguel Freitas in 2013.

According to the original README:

User registration and authentication is provided by a bitcoin-like network, so it is completely decentralized (does not depend on any central authority).
Post distribution uses kademlia DHT network and bittorrent-like swarms, both are provided by libtorrent.
Both Bitcoin and libtorrent versions included here are highly patched and do not interoperate with existing networks (on purpose).

Conception

The software design uses proof-of-work (PoW) to permanently store usernames (public keys) and message indexes in the blockchain. Mining is not required to create a new account; it is performed by service nodes that support the network by generating promotional messages. Also, the blockchain does not provide any coins for trading; it is solely used to offer decentralized storage for the network's usernames and messaging metadata.

To keep the blockchain as small as possible, all data exchanges occur over the BitTorrent protocol, using a modified version of the libtorrent library; this library is implemented as the local component of twister-core.

For more technical details about twister design, please read the original whitepaper

Development status

According to the announcement, on June 25, 2020, official project maintenance was discontinued, but the network remains active as it does not require centralized servers. Since 2021, development has been maintained by the community.

Requirements

• CPU architectures: amd64, armhf
• RAM: 512 Mb
• Disk: ~1Gb
  • build with dependencies: 529 Mb
  • profile data: ~512 Mb in 2025 (depending on subscriptions)
• Bandwidth: depends on the network activity, at this moment - about 2 Gb per month

User interface

Since twister-core has no GUI out of the box, it requires a web UI application called twister-html to interact using a web browser.

Install options

The simplest way to get started is to use the latest Flatpak bundle. To build from source, you will want to follow the instructions, depending on your system.

At this moment, there is no stable bundles available.

However, some experimental all-in-one builds, which include twister-core and twister-html for desktop use, are available from the community.

Flatpak

The best option if you want to try twister p2p from the sandbox, without compilling twister-core

• Flatpak, RC - in development: some features, such as mining and DHT, are not yet working properly

Debian / Ubuntu

Following DEB packages may work with preinstalled dependencies (e.g. libboost 1.74.0):

• amd64/deb - based on twister-core TA edition 1.2 (Jul 10, 2022)
• armhf/deb - based on twister-core TA edition 1.1 (Dec 25, 2021)

At this moment, the twister-core source code available in two options:

@miguelfreitas branch

Legacy version, released by Miguel Freitas in 2013.

According to the announcement, on June 25, 2020, official project maintenance was discontinued.

Website | Source code

@twisterarmy branch

Maintained by the community, it includes additional seeds and minor updates to make the codebase buildable on modern distributions.

It is 100% compatible with the main twister network (which was created by @miguelfreitas), and must remain compatible in the future.

Website | Source code

Fedora / Debian / Ubuntu

System dependencies

Debian / Ubuntu

sudo apt install git autoconf automake build-essential libtool\
                 libboost-all-dev libssl-dev libdb++-dev libminiupnpc-dev

Fedora

sudo dnf install git autoconf automake libtool make\
                 boost-devel openssl-devel libdb-cxx-devel miniupnpc-devel

Build steps

1. Build twister-core
   • sudo useradd -m twister - create new user with home directory (set password with sudo passwd twister)
   • su twister - login and navigate (with cd) into the home dir /home/twister to continue
   • git clone https://github.com/twisterarmy/twister-core.git
   • cd twister-core
   • ./autotool.sh
   • ./configure
     • ARM only: ./configure --with-boost-libdir=/usr/lib/arm-linux-gnueabihf --disable-sse2
   • make
2. Setup twister-html (web UI)
   • mkdir ~/.twister
   • echo -e "rpcuser=user\nrpcpassword=pwd\nrpcallowip=127.0.0.1" > ~/.twister/twister.conf
   • chmod 600 ~/.twister/twister.conf
   • git clone https://github.com/twisterarmy/twister-html.git ~/.twister/html
3. Launch twister
   • ./twisterd
   • open http://127.0.0.1:28332
     • enter user and pwd in the authorization dialog
   • create your account!

Upgrade from repository

Steps to upgrade your existing twister-core and twister-html from the repository sources:

1. stop running twisterd process to continue (if active, run pgrep twisterd to find)
   • sudo systemctl stop twister-service-name - if you're using systemd service
2. cd twister-core - navigate twister-core sources directory (which contains old twisterd binary)
3. git pull - grab latest updates
4. make - build new version
5. cd ~/.twister/html - navigate twister-html installation directory
6. git pull - grab latest twister-html changes
7. start twister-core process
   • sudo systemctl start twister-service-name - if you're using systemd service

##Install the development tools##

$ sudo yum groupinstall "Development Tools"

or

$ sudo yum install autoconf automake libtool

##Install the required dependencies##

Because the current version of Berkeley-db and Boost in CentOS 6 is kinds of old, so it's need to compile and install them from source.

First make the working directory.

$ cd ~
$ mkdir twister
$ mkdir twister/deps

Compile Berkeley-db

$ cd ~/twister
$ wget http://download.oracle.com/berkeley-db/db-4.8.30.tar.gz
$ tar -xzf db-4.8.30.tar.gz 
$ cd db-4.8.30/build_unix/
$ ../dist/configure --prefix=/home/XXX/twister/deps/ --enable-cxx
$ make
$ make install

(Please replace all the 'XXX' in this tutorial with your login name.)

Compile Boost

$ cd ~/twister
$ wget http://sourceforge.net/projects/boost/files/boost/1.55.0/boost_1_55_0.tar.bz2/download -O boost_1_55_0.tar.bz2
$ tar -jxf boost_1_55_0.tar.bz2 
$ cd boost_1_55_0
$ ./bootstrap.sh 
$ ./b2 --prefix=/home/XXX/twister/deps/ link=static runtime-link=static install

The Openssl that in the CentOS repo lacks of some components, so compile and install it from source also.

$ cd ~/twister
$ wget https://www.openssl.org/source/openssl-1.0.1i.tar.gz
$ tar -xzf openssl-1.0.1i.tar.gz 
$ cd openssl-1.0.1i
$ ./config --prefix=/home/XXX/twister/deps/ --openssldir=/home/XXX/twister/deps/openssl -fPIC shared
$ make
$ make install

##Compile the twister-core##

First install the Git tool if it doesn't install before.

$ sudo yum install git

Clone the twister-core source from Github.

$ cd ~/twister
$ git clone https://github.com/miguelfreitas/twister-core.git
$ cd twister-core
$ ./bootstrap.sh \
    --with-openssl=/home/XXX/twister/deps \
    --with-libdb=/home/XXX/twister/deps \
    --with-boost=/home/XXX/twister/deps
$ make

##Then the web part##

$ mkdir ~/.twister
$ git clone https://github.com/miguelfreitas/twister-html.git ~/.twister/html

##Start the twister daemon##

$ LD_LIBRARY_PATH=/home/XXX/twister/deps/lib/ /home/XXX/twister/twister-core/twisterd -daemon -rpcuser=user -rpcpassword=pwd -rpcallowip=127.0.0.1

Then open browser and goto http://127.0.0.1:28332/ ,enter "user" and "pwd" as the login username and password.

###References:###

• https://gist.github.com/janx/10465121

Prefer native mingw instructions, cygwin builds are known to have issues.

1. Install cygwin from http://cygwin.com/

2. During setup the following packages must be selected:

• autoconf
• automake
• libboost-devel
• libdb4.8-devel
• libtool
• gcc-g++
• git
• make
• openssl-devel
• tar
• (add here missing packages)

3. Download twister from github

git clone https://github.com/miguelfreitas/twister-core.git

cd twister-core

4. Compile twisterd

./bootstrap.sh

make

5. Create .twister directory and download HTML UI

mkdir ~/.twister

cd ~/.twister

git clone https://github.com/miguelfreitas/twister-html.git html

6. Now configure your username and password for the daemon:

echo -e "rpcuser=user\nrpcpassword=pwd" > ~/.twister/twister.conf

(NOTE: The username/password combo seems temporarily hardcoded. Also, if you receive an error message about "cygwin1.dll", add your cygwin path to the PATH environmental variable, eg. "C:\Cygwin64\bin".)

7. Run the daemon:

./twisterd -daemon -rpcuser=user -rpcpassword=pwd -rpcallowip=127.0.0.1

8. Open http://127.0.0.1:28332/index.html and use the user/pwd credentials.

Notes

• windows users are reporting problems to bootstrap the network (no connections). Try to manually add peer from one of the following seeder (try both "add peer" and "add dns"): seed.twister.net.co seed2.twister.net.co seed3.twister.net.co dnsseed.gombadi.com

• You may need to edit file /usr/lib/libdb_cxx-4.8.la to remove a wrong dependency (cygwin bug).

• Replace: dependency_libs=' /usr/lib/libdb-4.8.la -lpthread /usr/lib/gcc/i686-pc-cygwin/4.5.3/libstdc++.la'

• With: dependency_libs=' /usr/lib/libdb-4.8.la -lpthread'

• The following dlls are needed in order to run twisterd.exe (the file is inside .libs directory):

cygboost_filesystem-mt-1_53.dll cygboost_system-mt-1_53.dll cygcrypto-1.0.0.dll cygdb_cxx-4.8.dll cygssl-1.0.0.dll cygwin1.dll cygboost_program_options-mt-1_53.dll cygboost_thread-mt-1_53.dll cygdb-4.8.dll cyggcc_s-1.dll cygstdc++-6.dll cygz.dll

@wrewolf can build in clean Win XP SP3 x86 with latest cygwin (with upper note /), but modify original cygwin headers and insert defines from win32 headers. insert in file /usr/include/cygwin/in6.h lines from /usr/include/w32api/ws2ipdef.h

#define IPV6_HOPOPTS           1
#define IPV6_HDRINCL           2
#define IPV6_UNICAST_HOPS      4
#define IPV6_MULTICAST_IF      9
#define IPV6_MULTICAST_HOPS    10
#define IPV6_MULTICAST_LOOP    11
#define IPV6_ADD_MEMBERSHIP    12
#define IPV6_JOIN_GROUP        IPV6_ADD_MEMBERSHIP
#define IPV6_DROP_MEMBERSHIP   13
#define IPV6_LEAVE_GROUP       IPV6_DROP_MEMBERSHIP
#define IPV6_DONTFRAG          14
#define IPV6_PKTINFO           19
#define IPV6_HOPLIMIT          21
#define IPV6_PROTECTION_LEVEL  23
#define IPV6_RECVIF            24
#define IPV6_RECVDSTADDR       25
#define IPV6_CHECKSUM          26
#define IPV6_V6ONLY            27
#define IPV6_IFLIST            28
#define IPV6_ADD_IFLIST        29
#define IPV6_DEL_IFLIST        30
#define IPV6_UNICAST_IF        31
#define IPV6_RTHDR             32
#define IPV6_RECVRTHDR         38
#define IPV6_TCLASS            39
#define IPV6_RECVTCLASS        40

And build without any modifycation sources

./bootstrap.sh
make V=1

(please help improving this page)

Pre-Configured Gitian VirtualBox image

A pre-configured VirtualBox Gitian image based on Ubuntu 12.04 LTS x86_64 + gitian + twister sources is now available for download (3.2 GB file):

ubuntu_12.04_amd64_gitian_twister.ova (download from Mega)

ubuntu_12.04_amd64_gitian_twister.ova (download from Dropbox)

ubuntu_12.04_amd64_gitian_twister.ova (download from Google Drive)

The md5sum of the file is:

48f62fce1ba105cdcd172b8de6a2466a ubuntu_12.04_amd64_gitian_twister.ova

The Ubuntu desktop contains a simple README text file with very basic gitian instructions on building both 32 and 64-bits native Windows executables (using mingw, not cygwin).

One just have to boot the VirtualBox and execute a couple of commands to create an updated twisterd.exe from twister-core github repository.

For more information on the building process refer to Gitian documentation.

Create your Gitian build without VirtualBox (Ubuntu required)

For the most up-to-date instructions see: [[https://github.com/miguelfreitas/twister-core/tree/master/contrib/gitian-descriptors]] and [[https://github.com/miguelfreitas/twister-core/blob/master/doc/release-process.md]]

You need the right hardware: you need a 64-bit-capable CPU with hardware virtualization support (Intel VT-x or AMD-V). Not all modern CPUs support hardware virtualization.

You probably need to enable hardware virtualization in your machine's BIOS.

You need to be running a recent version of 64-bit-Ubuntu, and you need to install several prerequisites:

sudo apt-get install ruby apache2 git apt-cacher-ng python-vm-builder qemu-kvm

Sanity checks:

sudo service apt-cacher-ng status  # Should return apt-cacher-ng is running
ls -l /dev/kvm   # Should show a /dev/kvm device

Once you've got the right hardware and software:

git clone git://github.com/miguelfreitas/twister-core.git
git clone git://github.com/devrandom/gitian-builder.git
mkdir gitian-builder/inputs

# Create base images
cd gitian-builder
bin/make-base-vm --suite precise --arch i386
bin/make-base-vm --suite precise --arch amd64
cd ..

# Get inputs (see doc/release-process.md for exact inputs needed and where to get them)
cd gitian-builder/inputs
wget 'https://www.openssl.org/source/openssl-1.0.1h.tar.gz'
wget 'http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz'
wget 'http://zlib.net/zlib-1.2.8.tar.gz'
wget 'https://downloads.sourceforge.net/project/boost/boost/1.55.0/boost_1_55_0.tar.bz2'
wget 'https://svn.boost.org/trac/boost/raw-attachment/ticket/7262/boost-mingw.patch' -O boost-mingw-gas-cross-compile-2013-03-03.patch
wget 'https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.bz2'
wget 'https://github.com/mingwandroid/toolchain4/archive/10cc648683617cca8bcbeae507888099b41b530c.tar.gz'
wget 'http://www.opensource.apple.com/tarballs/cctools/cctools-809.tar.gz'
wget 'http://www.opensource.apple.com/tarballs/dyld/dyld-195.5.tar.gz'
wget 'http://www.opensource.apple.com/tarballs/ld64/ld64-127.2.tar.gz'
wget 'https://github.com/theuni/libdmg-hfsplus/archive/libdmg-hfsplus-v0.1.tar.gz'
wget 'http://llvm.org/releases/3.2/clang+llvm-3.2-x86-linux-ubuntu-12.04.tar.gz' -O clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz
cd ..
./bin/gbuild ../twister-core/contrib/gitian-descriptors/boost-linux.yml
mv build/out/boost-*.zip inputs/
./bin/gbuild ../twister-core/contrib/gitian-descriptors/deps-linux.yml
mv build/out/twister-deps-*.zip inputs/
./bin/gbuild ../twister-core/contrib/gitian-descriptors/boost-win.yml
mv build/out/boost-*.zip inputs/
./bin/gbuild ../twister-core/contrib/gitian-descriptors/deps-win.yml
mv build/out/twister-deps-*.zip inputs/
./bin/gbuild ../twister-core/contrib/gitian-descriptors/protobuf-win.yml
mv build/out/protobuf-*.zip inputs/
./bin/gbuild ../twister-core/contrib/gitian-descriptors/gitian-osx-native.yml
mv build/out/osx-*.tar.gz inputs/
./bin/gbuild ../twister-core/contrib/gitian-descriptors/gitian-osx-depends.yml
mv build/out/osx-*.tar.gz inputs/
./bin/gbuild ../twister-core/contrib/gitian-descriptors/gitian-osx-qt.yml
mv build/out/osx-*.tar.gz inputs/

Build twister for win 32 and 64-bits:

./bin/gbuild --commit twister-core=HEAD ../twister-core/contrib/gitian-descriptors/gitian-win.yml

Q/A

What is this?
This page contains instructions on how to build twister with Visual Studio 2010.

Is this relevant for me if I want to run twister on Windows?
No. If you just need to run twister, please get the twister build based on Cygwin as it is more stable at this time. This is for people who are familiar with Visual Studio and want to develop/debug twister there.

Does it really work?
Not exactly. It builds and runs. You can open the GUI and start using it. It crashes after a while though. But this is just a matter of debugging it. From what I see, most of the problems are related to STL usage and MS specifics in this area.

Why don't you make it work?
I don't have as much time to work on this as I would like. I would greatly appreciate if a younger C++ padawan would take a look at it and fix it. If not, I will probably fix it myself eventually.

How to build twister with VS 2010 (VC 10)

1.) Download twister-core, put it into c:\twister\twister-core

2.) Download OpenSSL 1.0.1c, put it into c:\twister\twister-core\openssl-1.0.1c (see below on how to build it)

3.) Download Boost 1.43.0, put it into c:\twister\twister-core\boost_1_43_0 (see below on how to build it)

4.) Download Berkeley DB 4.8.30.NC, put it into c:\twister\twister-core\db-4.8.30.NC (see below on how to build it)

5.) Download ittypes.h (from https://code.google.com/p/msinttypes) and put it into c:\twister\vcbuild\include

6.) Create the environmental variable TWISTER_CORE and set it to c:\twister\twister-core

7.) Create the environmental variable LIBTORRENT and set it to c:\twister\twister-core\libtorrent

8.) Create the environmental variable OPENSSL and set it to c:\twister\twister-core\openssl-1.0.1c

9.) Create the environmental variable BOOST_ROOT and set it to c:\twister\twister-core\boost_1_43_0

10.) Create the environmental variable BERKELEY_DB and set it to c:\twister\twister-core\db-4.8.30.NC

11.) Create a Win32 Console Application project named "twisterd" in c:\twister\twister-core\vcbuild. Under Additional options, select Empty project.

12.) Add source files to the project in the following way:

[Source Files]
  (stuff from c:\twister\twister-core\src)
  addrman.cpp
  alert.cpp
  bitcoind.cpp
  bitcoinrpc.cpp
  bloom.cpp
  chainparams.cpp
  checkpoints.cpp
  core.cpp
  crypter.cpp
  db.cpp
  hash.cpp
  init.cpp
  key.cpp
  keystore.cpp
  leveldb.cpp
  main.cpp
  net.cpp
  netbase.cpp
  noui.cpp
  protocol.cpp
  rpcblockchain.cpp
  rpcdump.cpp
  rpcmining.cpp
  rpcnet.cpp
  rpcrawtransaction.cpp
  rpcwallet.cpp
  script.cpp
  scrypt-sse2.cpp
  scrypt.cpp
  sync.cpp
  twister.cpp
  twister_utils.cpp
  txdb.cpp
  util.cpp
  version.cpp
  wallet.cpp
  walletdb.cpp
  [leveldb] (stuff from c:\twister\twister-core\src\leveldb)
    [db]
      builder.cc
      c.cc
      dbformat.cc
      db_impl.cc
      db_iter.cc
      filename.cc
      log_reader.cc
      log_writer.cc
      memtable.cc
      repair.cc
      table_cache.cc
      version_edit.cc
      version_set.cc
      write_batch.cc
    [helpers]
      [memenv]
        memenv.cc
    [port]
      port_win.cc
    [table]
      block.cc
      block_builder.cc
      filter_block.cc
      format.cc
      iterator.cc
      merger.cc
      table.cc
      table_builder.cc
      two_level_iterator.cc
    [util]
      arena.cc
      bloom.cc
      cache.cc
      coding.cc
      comparator.cc
      crc32c.cc
      env.cc
      env_win.cc
      filter_policy.cc
      hash.cc
      histogram.cc
      logging.cc
      options.cc
      status.cc
      testharness.cc
      testutil.cc
  [libtorrent] (stuff from c:\twister\twister-core\libtorrent\src)
    alert.cpp
    alert_manager.cpp
    allocator.cpp
    asio.cpp
    asio_ssl.cpp
    assert.cpp
    bandwidth_limit.cpp
    bandwidth_manager.cpp
    bandwidth_queue_entry.cpp
    bloom_filter.cpp
    broadcast_socket.cpp
    bt_peer_connection.cpp
    chained_buffer.cpp
    connection_queue.cpp
    ConvertUTF.cpp
    create_torrent.cpp
    disk_buffer_holder.cpp
    disk_buffer_pool.cpp
    disk_io_thread.cpp
    entry.cpp
    enum_net.cpp
    error_code.cpp
    escape_string.cpp
    file.cpp
    file_pool.cpp
    file_storage.cpp
    gzip.cpp
    hasher.cpp
    http_connection.cpp
    http_parser.cpp
    http_seed_connection.cpp
    http_stream.cpp
    http_tracker_connection.cpp
    i2p_stream.cpp
    identify_client.cpp
    instantiate_connection.cpp
    ip_filter.cpp
    ip_voter.cpp
    lazy_bdecode.cpp
    logger.cpp
    lsd.cpp
    lt_trackers.cpp
    magnet_uri.cpp
    metadata_transfer.cpp
    natpmp.cpp
    packet_buffer.cpp
    parse_url.cpp
    peer_connection.cpp
    pe_crypto.cpp
    piece_picker.cpp
    policy.cpp
    puff.cpp
    random.cpp
    rsa.cpp
    rss.cpp
    session.cpp
    session_impl.cpp
    settings.cpp
    sha1.cpp
    smart_ban.cpp
    socket_io.cpp
    socket_type.cpp
    socks5_stream.cpp
    stat.cpp
    storage.cpp
    string_util.cpp
    thread.cpp
    time.cpp
    timestamp_history.cpp
    torrent.cpp
    torrent_handle.cpp
    torrent_info.cpp
    tracker_manager.cpp
    udp_socket.cpp
    udp_tracker_connection.cpp
    upnp.cpp
    utf8.cpp
    utp_socket_manager.cpp
    utp_stream.cpp
    ut_metadata.cpp
    ut_pex.cpp
    web_connection_base.cpp
    web_peer_connection.cpp
    GeoIP.c
    mpi.c
    [kademlia]
      dht_get.cpp
      dht_tracker.cpp
      find_data.cpp
      logging.cpp
      node.cpp
      node_id.cpp
      refresh.cpp
      routing_table.cpp
      rpc_manager.cpp
      traversal_algorithm.cpp

13.) Open properties for libtorrent\alert.cpp, go to C/C++ / Output Files, set Object File Name to $(IntDir)libtorrent_alert.obj

14.) Open properties for leveldb\util\bloom.cc, go to C/C++ / Output Files, set Object File Name to $(IntDir)leveldb_util_bloom.obj

15.) Open properties for leveldb\util\hash.cc, go to C/C++ / Output Files, set Object File Name to $(IntDir)leveldb_util_hash.obj

16.) Apply patches to source files (see below)

17.) Open the project Property Pages. (The instructions that follow are for the Win32 platform and debug configuration.)

18.) Add the following folders to Additional Include Directories under C/C++ / General:

$(LIBTORRENT)\include
$(BOOST_ROOT)
$(OPENSSL)\inc32
$(TWISTER_CORE)\vcbuild\include
$(TWISTER_CORE)\src
$(TWISTER_CORE)\src\leveldb
$(TWISTER_CORE)\src\leveldb\include
$(TWISTER_CORE)\src\leveldb\helpers
$(BERKELEY_DB)\build_windows

19.) Add the following definitions to Preprocessor Definitions under C/C++ / Preprocessor:

WIN32
_DEBUG
_CONSOLE
WIN32_LEAN_AND_MEAN
_WIN32_WINNT=0x0501
_MSC_VER=1600
_CRT_SECURE_NO_DEPRECATE
BOOST_ASIO_HASH_MAP_BUCKETS=1021
BOOST_FILESYSTEM_VERSION=2
WITH_SHIPPED_GEOIP_H
BOOST_ASIO_SEPARATE_COMPILATION
BOOST_ASIO_ENABLE_CANCELIO
TORRENT_USE_OPENSSL
NOMINMAX
__PRETTY_FUNCTION__=__FUNCTION__
BOOST_HAS_STDINT_H
LEVELDB_PLATFORM_WINDOWS
OS_WIN

20.) Add the following folders to Additional Library Directories under Linker / General:

$(BOOST_ROOT)\stage\lib
$(OPENSSL)\out32
$(BERKELEY_DB)\build_windows\Win32\Debug

21.) Add the following libs to Additional Dependencies under Linker / Input:

libeay32.lib
ssleay32.lib
shlwapi.lib
libdb48d.lib

22.) Copy libdb48d.dll from c:\twister\twister-core\db-4.8.30.NC\build_windows\Win32\Debug to c:\twister\twister-core\vcbuild\Debug

23.) Hit F5 and keep your fingers crossed (this will take some time)

Building OpenSSL 1.0.1c

1.) Download and install ActivePerl (http://www.activestate.com/activeperl/downloads)

2.) Download and install NASM (http://www.nasm.us)

3.) Create the environmental variable NASM and set it to the NASM folder, e.g. C:\Program Files (x86)\NASM

4.) Start Visual Studio Command Prompt

5.) Run the following commands:

cd C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin vcvars32.bat cd c:\twister\twister-core\openssl-1.0.1c perl Configure VC-WIN32 --prefix="%OPENSSL%" "%NASM%"\nasmpath.bat call ms\do_nasm nmake -f ms\nt.makdir

This produces exe and lib files in c:\twister\twister-core\openssl-1.0.1c\out32, and header files in c:\twister\twister-core\openssl-1.0.1c\inc32.

Note: Making vcvars32.bat work is a real pain in the a**. It contains a lot of statements containing environmental variables surrounded by quotes such as set "PATH=%WindowsSdkDir%bin\NETFX 4.0 Tools;%WindowsSdkDir%bin;%PATH%" and if exist "%VSINSTALLDIR%Team Tools\Performance Tools" set .... The problem occurs when the value of one of these variables also contains quotes. If you get an error saying something like "\Microsoft\Microsoft was unexpected at this time", your best bet is to make sure that the value of the environmental var PATH does not contain any quotes.

Building Boost 1.43.0

1.) Start Visual Studio Command Prompt

2.) Run the following commands:

cd C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin vcvars32.bat cd c:\twister\twister-core\boost_1_43_0 bootstrap.bat bjam toolset=msvc-10.0 --build-type=complete stage

This process takes some time. It produces lib files in c:\twister\twister-core\boost_1_43_0\stage\libs.

Note: If vcvars32.bat and/or bootstrap.bat fails, see the comment above on why this happens and how to fix it.

Building Berkeley DB 4.8.30.NC

Coming soon...

(Patches also coming soon.)

Boost C++

If the twisterd build fails because your system repository has a newer version of libboost that is incompatible with the current twister-core API, follow these steps to build a custom (legacy) version from source:

1. wget https://archives.boost.io/release/1.83.0/source/boost_1_83_0.tar.gz - get specified version (replace 1_83_0 with your value)
   • compare the output of sha256sum boost_1_83_0.tar.gz which should be c0685b68dd44cc46574cce86c4e17c0f611b15e195be9848dfd0769a0a207628
2. tar -xzf boost_1_83_0.tar.gz - unarchive the sources
3. cd boost_1_83_0 - navigate the sources directory
4. ./bootstrap.sh --prefix=/path/to/boost-build - replace /path/to/boost-build with your value
   • optionally, you may skip the --prefix= argument, when installing the boost into default location, but it is not recommended as may overwrite your existing system version installed by package manager!
5. ./b2 install - build and install to the --prefix= destination (optionally provide -j argument if you want to specify the number of CPU threads)

When the boost build is complete, configure the twister-core using --with-boost=/path/to/boost-build argument, for example:

cd twister-core
./configure --with-boost=/path/to/boost-build

• then build twisterd with make

Berkeley DB

The current version of twister-core should be compatible with recent system versions of Berkeley DB (also known as bdb, libdb), at least on Fedora 41, which provides version 5.3.28 from its repository.

If you want to build a specific version of libdb from source, use the following steps:

1. wget https://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz - get legacy version (or replace 4.8.30.NC with your value)
   • compare the output of sha256sum db-4.8.30.NC.tar.gz which should be 12edc0df75bf9abd7f82f821795bcee50f42cb2e5f76a6a281b85732798364ef
2. tar -xzf db-4.8.30.NC.tar.gz - unarchive the sources
3. cd db-4.8.30.NC - navigate the sources directory
   • wget https://gist.githubusercontent.com/LnL7/5153b251fd525fe15de69b67e63a6075/raw/7778e9364679093a32dec2908656738e16b6bdcb/clang.patch - download this patch when building with clang and C++11 (details)
   • patch -p2 < clang.patch - apply a patch
4. cd build_unix - navigate to this directory if you are building on Linux
5. ../dist/configure --prefix=/path/to/libdb-build --enable-cxx - replace /path/to/libdb-build with your value
   • optionally, you may skip the --prefix= argument, when installing the libdb into default location, but it is not recommended as may overwrite your existing system version installed by package manager!
   • if you see WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM - as a temporary solution, append CFLAGS=-Wno-error=implicit-function-declaration to the configure command above
6. make - build (optionally provide -j argument if you want to specify the number of CPU threads)
7. make install - install the build to the --prefix= destination (/path/to/libdb-build)

When our libdb version is ready to use, configure the twister-core using --with-libdb=/path/to/libdb-build argument, for example:

cd twister-core
./configure --with-libdb=/path/to/libdb-build

• then build twisterd with make

To build libtorrent on 32b Ubuntu:

sudo apt-get install autoconf libtool libssl-dev libboost-all-dev
cd libtorrent
./bootstrap.sh --with-boost-libdir=/usr/lib/i386-linux-gnu
./configure --enable-logging --enable-debug --enable-dht --with-boost-libdir=/usr/lib/i386-linux-gnu

To build libtorrent on 64b Ubuntu:

sudo apt-get install autoconf libtool libssl-dev libboost-all-dev
cd libtorrent
./bootstrap.sh --with-boost-libdir=/usr/lib/x86_64-linux-gnu/
./configure --enable-logging --enable-debug --enable-dht --with-boost-libdir=/usr/lib/x86_64-linux-gnu/

Launch

The twister node is usually launched from the same or a separate system user, and never from root.

If you want to run it from a separate user, just create the user with useradd -m twister, then log in and build the twister-core inside this directory to have valid permissions out of the box.

To launch, run following command:

./twister

• see CLI arguments for available startup options

CLI arguments

If you want to run twisterd using a proxy or bind the connection to a specified network interface, consider using the following options:

-?                                     This help message
-conf=file                             Specify configuration file (default: ~/.twister/twister.conf)
-pid=file                              Specify pid file (default: twisterd.pid)
-gen                                   Generate coins (default: 0)
-datadir=dir                           Specify data directory
-htmldir=dir                           Specify HTML directory to serve (default: data/html)
-dbcache=n                             Set database cache size in megabytes (default: 25)
-timeout=n                             Specify connection timeout in milliseconds (default: 5000)
-proxy=ip:port                         Connect through socks proxy
-socks=n                               Select the version of socks proxy to use (4-5, default: 5)
-tor=ip:port                           Use proxy to reach tor hidden services (default: same as -proxy)
-dns                                   Allow DNS lookups for -addnode, -seednode and -connect
-port=port                             Listen for connections on port (default: 28333 or testnet: 18333)
-maxconnections=n                      Maintain at most n connections to peers (default: 125)
-addnode=ip                            Add a node to connect to and attempt to keep the connection open
-connect=ip                            Connect only to the specified node(s)
-seednode=ip                           Connect to a node to retrieve peer addresses, and disconnect
-externalip=ip                         Specify your own public address
-onlynet=net                           Only connect to nodes in network net (IPv4, IPv6 or Tor)
-multiconnperip                        Enable libtorrent multiple connections per ip (default: 0)
-discover                              Discover own IP address (default: 1 when listening and no -externalip)
-checkpoints                           Only accept block chain matching built-in checkpoints (default: 1)
-listen                                Accept connections from outside (default: 1 if no -proxy or -connect)
-bind=addr                             Bind to given address and always listen on it. Use [host]:port notation for IPv6
-dnsseed                               Find peers using DNS lookup (default: 1 unless -connect)
-banscore=n                            Threshold for disconnecting misbehaving peers (default: 100)
-bantime=n                             Number of seconds to keep misbehaving peers from reconnecting (default: 86400)
-maxreceivebuffer=n                    Maximum per-connection receive buffer, n*1000 bytes (default: 5000)
-maxsendbuffer=n                       Maximum per-connection send buffer, n*1000 bytes (default: 1000)
-upnp                                  Use UPnP to map the listening port (default: 1 when listening)
-paytxfee=amt                          Fee per KB to add to transactions you send
-testnet                               Use the test network
-debug                                 Output extra debugging information. Implies all other -debug* options
-debugnet                              Output extra network debugging information
-logtimestamps                         Prepend debug output with timestamp
-shrinkdebugfile                       Shrink debug.log file on client startup (default: 1 when no -debug)
-printtoconsole                        Send trace/debug info to console instead of debug.log file
-regtest                               Enter regression test mode, which uses a special chain in which blocks can be solved instantly. This is intended for regression testing tools and app development.
-rpcuser=user                          Username for JSON-RPC connections
-rpcpassword=pw                        Password for JSON-RPC connections
-rpcport=port                          Listen for JSON-RPC connections on port (default: 28332 or testnet: 18332)
-rpcallowip=ip                         Allow JSON-RPC connections from specified IP address
-rpcconnect=ip                         Send commands to node running on ip (default: 127.0.0.1)
-rpcthreads=n                          Set the number of threads to service RPC calls (default: 10)
-public_server_mode                    Limit JSON-RPC execution to public/safe commands only.
-blocknotify=cmd                       Execute command when the best block changes (%s in cmd is replaced by block hash)
-walletnotify=cmd                      Execute command when a wallet transaction changes (%s in cmd is replaced by TxID)
-alertnotify=cmd                       Execute command when a relevant alert is received (%s in cmd is replaced by message)
-upgradewallet                         Upgrade wallet to latest format
-keypool=n                             Set key pool size to n (default: 100)
-rescan                                Rescan the block chain for missing wallet transactions
-salvagewallet                         Attempt to recover private keys from a corrupt twisterwallet.dat
-checkblocks=n                         How many blocks to check at startup (default: 500, 0 = all)
-checklevel=n                          How thorough the block verification is (0-4, default: 3)
-txindex                               Maintain a full transaction index (default: 0)
-loadblock=file                        Imports blocks from external blk000??.dat file
-reindex                               Rebuild block chain index from current blk000??.dat files
-par=n                                 Set the number of script verification threads (up to 16, 0 = auto, 0 = leave that many cores free, default: 0)

-blockminsize=n                        Set minimum block size in bytes (default: 0)
-blockmaxsize=n                        Set maximum block size in bytes (default: 250000)
-blockprioritysize=n                   Set maximum size of high-priority/low-fee transactions in bytes (default: 27000)

-rpcssl                                Use OpenSSL (https) for JSON-RPC connections
-rpcsslcertificatechainfile=file.cert  Server certificate file (default: server.cert)
-rpcsslprivatekeyfile=file.pem         Server private key (default: server.pem)
-rpcsslciphers=ciphers                 Acceptable ciphers (default: TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!AH:!3DES:@STRENGTH)

Profile data

Typically, user data is created upon the first launch and stored in the home directory of the user who runs the twisterd process:

/home/username/.twister

~/.twister contents

• /blocks, /chainstate - blockchain data
• /database - application state database
• /html - default location for the twister-html web UI, may not be here if twisterd has been launched with the -htmldir argument
• debug.log - contains recent RPC requests
• peers.dat - the known peers database, along with the current network state, could be useful for initial connection to old peers without working DNS seeds (by copying it into the new twister-core profile)
• twister.conf - stores twisterd launch options used instead of some CLI arguments
• twisterwallet.dat - contains user private keys, it is same as wallet.dat in Bitcon, keep in safe place!
• .lock - prevents twisterd from being launched twice at the same time

Backup

Similar to a Bitcoin, which uses ~/.bitcoin/wallet.dat, twister manages your accounts using the ~/.twister/twisterwallet.dat file.

To backup a specified username (public key) in twister, you should know its password (private key) that was previously registered on the twister blockchain, which was also displayed in the success alert when the account was created.

Web UI

To export your private key for an active user:

1. navigate to the Home page
2. select from the main menu Setup account
3. press Secret Key button
4. your private key will be displayed below, copy it to a safe place.

Files to backup

An additional backup of the profile folder is not really required, as the blockchain and other data can be downloaded from the network during the initialization of a new wallet.

Optionally, you may archive the entire .twister folder or just copy twisterwallet.dat to use your wallet on another twister node without having to download the blockchain or import the public/private key pair in the web UI.

How to store backup

Similar to Bitcoin, you can keep your profile data in digital form (by using KeePassXC) or you even can simply print the public/private key pair on paper.

Remote node running is useful in cases where you want to access twister from a mobile device or support the network by launching a public peer with a web UI.

Setup with CLI tool

The simplest way to get started on Debian systems is by using the twister-cli-installer script. It provides guiding questions that helps you to build the twister-core, twister-html, and set up a self-signed TLS certificate for secure connection.

Manual setup

Login to the remote server as root with SSH:

ssh root@hostname

Firewall configuration

Open 28332 port for the external access:

ufw allow from EXPECTED_REMOTE_IP to any port 28332

• replace EXPECTED_REMOTE_IP with your value
• the port 28332 may be different if you are running twisterd with a custom -rpcport argument value

Enable the firewall:

ufw enable

Check firewall status:

ufw status

System

Create system user

Create separated system user for twisterd process:

useradd -m twister

Create systemd unit

It is useful to operate remote twisterd process with systemd. Let's create its configuration file:

[Unit]
After=network.target

[Service]
Type=simple
User=twister
Group=twister
# this option is not necessary
WorkingDirectory=/home/twister
# provide real path or additional arguments here if wanted
ExecStart=/home/twister/twister-core/twisterd
# where to save the logs
StandardOutput=file:/home/twister/twisterd-debug.log
StandardError=file:/home/twister/twisterd-error.log
# you may remove this option to capture the failure event
Restart=on-failure

[Install]
WantedBy=multi-user.target

• systemctl daemon-reload - apply systemd update
• systemctl enable twister - launch twisterd on system startup

Now login to the system user created:

su twister

• navigate to the home directory by using the cd command without arguments

User environment setup

Install twister-core

Build twisterd from the twister-core sources inside this home, or simply copy the twisterd binary to /home/twister/twister-core/twisterd directory (as expected in the ExecStart option above).

chmod +x path/to/twisterd

Generate self-signed certificate

openssl req -x509\
            -newkey rsa:4096\
            -keyout /home/twister/.twister/key.pem\
            -out /home/twister/.twister/cert.pem\
            -days 365 -nodes

• with the example above, you will want to create a new certificate after 365 days!

Setup RPC authorization

Edit your /home/twister/.twister/twister.conf file:

rpcallowip=EXPECTED_REMOTE_IP
rpcuser=RPC_USER_NAME
rpcpassword=RPC_PASSWORD
rpcsslcertificatechainfile=/home/twister/.twister/cert.pem
rpcsslprivatekeyfile=/home/twister/.twister/key.pem
rpcssl=1

• replace EXPECTED_REMOTE_IP, RPC_USER_NAME, RPC_PASSWORD with your values;
• if you have a dynamic client IP, you may use * as the rpcallowip value, but this option is not recommended without additional firewall rules!
• when using a connection setup with SSL enabled, in some cases, try launching twisterd with the -rpcssl argument instead of defining it in the twister.conf file.

Now exit from the current user session:

exit

Launch twisterd

From the current root session, launch our newly created systemd service (which will start the twisterd process from the twister user):

systemctl start twister

• systemctl status twister - check if everything is working

Testing remote connection

Web UI

1. Open following address in your browser: https://SERVER_IP:28332/
   • or use http if the TLS certificate is not in use
2. Enter your RPC_USER_NAME and RPC_PASSWORD on the authorization dialog

RPC API

For the RPC API, use the same RPC_USER_NAME and RPC_PASSWORD for SERVER_IP:28332 as you would when using the HTTP protocol; also, provide the actual scheme to connect.

Final tips

Setup host alias

You can access the remote node using hostname alias instead of the IP address by appending the following line to /etc/hosts:

REMOTE_HOST_IP twister

• replace REMOTE_HOST_IP with actual IP, then open web UI with http://twister:28332

SSH login without password

You can simply log in to the remote host account using your SSH key instead of the system user password.

Run from the client host:

ssh-copy-id twister@REMOTE_HOST_IP

• where REMOTE_HOST_IP is the host IP of your remote twister node
• provide the -p argument if your host is configured to use a non-default SSH port (other than 22)

Improve security

Use strong RPC credentials

Do not use the default user:pwd for the remote twister node. Consider using random, strong RPC login credentials when any of your ports are open to external connections.

Run public peer with secure options

If you do not plan to use a remote node with an account, run twisterd with the -public_server_mode argument to restrict unsafe commands (e.g., wallet operations).

Install fail2ban

If you are running a public peer server with remote control through SSH, consider installing the fail2ban daemon, which will block connections to port 22 (or another port) after an invalid login attempt (for 15 minutes by default).

Randomize ports

Use a non-default SSH port to confuse bots and prevent your logs from growing.

Prioritize using a firewall

Configure firewall/iptables rules to restrict access by using only whitelisted values.

Remote host administration

Refer to the Administration page for a useful collection of CLI recipes!

Disk

Custom profile location

By default, twisterd will create a profile directory on the same drive where it is launched.

If you are using a single-board device to run twisterd with default startup options, this could lead to many overwrite operations on the built-in SSD storage and reduce its lifespan. To resolve this, launch ./twisterd with the -datadir=DATA argument, where DATA is an absolute path to your custom profile data location. For example, you can specify a location on an external drive (like a USB flash drive) mounted that is easy to replace or extend in size.

External drive

1. connect your drive then run parted -l or fdisk -l to find its location (e.g. /dev/sda or /dev/mmcblk0)
2. navigate to the partition manager for this device using the command parted /dev/sda
3. mklabel gpt - set partition format
4. mkpart logical ext4 0% 100% - to use all available space
5. print free check if everything fine here
6. quit - save and exit
7. mkfs -t ext4 /dev/sda1 - format partition created (replace /dev/sda1 with your value)
8. lsblk -o PATH,SIZE,RO,TYPE,MOUNTPOINT,UUID,MODEL - get partition UUID
9. nano /etc/fstab - persist on startup

# <file system>                      <mount point> <type> <options>                    <dump> <pass>
...
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxx /mnt/sda1     ext4  defaults,noatime,rw,user,auto 0      2

• save and exit

10. mv /home/twister/.twister /mnt/sda1/.twister - move profile data from the root as mounded by this user
11. chown -R twister:twister /mnt/sda1/.twister - since you have copied files as the root, make sure the twister user has permission to access this directory when launching twisterd
12. ./twisterd -datadir=/mnt/sda1/.twister - now run twisterd as its regular user

Free additional space

Before continuing with the options below, try to search for large files inside the user profile under which you are running the twister-core service:

1. su twister - login to your twister-core account
2. find ~/.twister -type f -size +50M - search for files larger than 50 MB
3. watch for .log files in results to clean up and prioritize their management
4. repeat steps 2-3 for the /var/log/* location

Clean up the profile cache

In this case, you can try to remove the contents of the ~/.twister/swarm folder (after running systemctl stop twister-core), as it may grow over time and take up a lot of disk space on older service nodes.

Memory

Extend memory by using a swap file

1. fallocate -l 1G /swapfile - creates new swapfile in the filesystem root
2. chmod 600 /swapfile - give the valid permissions
3. mkswap /swapfile - format
4. swapon /swapfile - enable
5. free -h or swapon --show or htop - make sure the memory has been extended

# <file system>   <mount point>   <type>   <options>   <dump> <pass>
/swapfile         none            swap     sw          0      0

If the swapfile is no longer in use, you can safely remove it:

1. swapoff /swapfile - await to continue
2. free -h or swapon --show or htop - make sure the swapfile was disabled
3. rm /swapfile - free disk by removing swapfile created in the previous steps

Limit memory usage for the twisterd process

To forcefully restrict memory resources for twisterd process, add the following options to the systemd service:

[Service]
MemoryHigh=256M
MemoryMax=512M
MemorySwapMax=512M

• save and exit
• systemctl daemon-reload - do not forget to reload systemd configuration
• systemctl restart twister - restart service (where twister is the actual service name)

Useful CLI tools

• sensors (apt install lm-sensors) - show current CPU temperature (useful for miners)
• htop, btop - system totals monitor
• df -h - show disk usage summary
• find / -type f -size +50M - find large files (provide your path and size)

Service peer

If you are running a public peer or just using twister-core on a machine with limited resources, consider following these recommendations:

Peering

Do not follow too many accounts, especially RSS bots that produce massive feeds, as every message in the twister peer-to-peer network is a separate torrent that requires hash management.

Mining

If you're a network supporter and mining blocks on a powerful machine, take a look at the twistead tool, which provides the --latency option that hibernates the miner for a specified time in seconds as soon as a new block has been mined. If someone mines a block after the timeout expires, the daemon will continue to sleep.

Yggdrasil is a decentralized IPv6 network that offers an additional layer of privacy for peer-to-peer connections.

According to the original README:

Yggdrasil is an early-stage implementation of a fully end-to-end encrypted IPv6 network. It is lightweight, self-arranging, supported on multiple platforms and allows pretty much any IPv6-capable application to communicate securely with other Yggdrasil nodes. Yggdrasil does not require you to have IPv6 Internet connectivity - it also works over IPv4.

Usage purposes

Yggdrasil can be useful in situations when you want to:

• hide your real IP address from other (twister) peers;
• bypass ISPs or global firewalls in censored areas;
• build a custom route to the VPN, I2P or Tor proxy;
• remote access to your home node over NAT;
• create a permanent IPv6 address in the 0200::/7 range (based on the private key) to set up a remote peer or firewall for whitelisting;
• run a public peer (accessible to other twister nodes connected to the Yggdrasil network) without needing a dedicated Internet IP address and port forwarding.

IPv6 support

If you are using twister-core from the twisterarmy branch, no additional steps are needed, as it already has the IPv6 enabled out of the box.

However, if you are using the twister-core from the miguelfreitas branch, you must configure it with the option --enable-ipv6 (see ./configure --help for details), then rebuild twisterd.

Usage examples

Connect all peers

By launching twisterd without additional arguments, you will be able to connect to all network interfaces, including the Yggdrasil one.

If you don't see any connections from the 0200::/7 range yet, visit the community website and find any peer from the list. After that, copy its [host]:port into the Force connection to peer field on the web UI Network page.

./twisterd -externalip=[HOST]

Connect specified peer only

Run twisterd with -connect=[HOST]:PORT argument, where the HOST:PORT is the target host you want to connect to:

./twisterd -connect=[300:17a8:aabf:108f::33]:28333

Connect some network only

By default, twisterd accepts connections from all available network interfaces. This option can be helpful in situations where you are using some proxy that does not support certain address families, such as how yggstack does not support IPv4.

To use only the IPv6 network family, launch twisterd with the -onlynet=IPv6 flag:

./twisterd -onlynet=IPv6

• IPv4|IPv6|Tor - see also ./twisterd --help

Connect with proxy

yggstack

yggstack is a proxy server for Yggdrasil that allows you to use this network without installing the full node. It is especially useful in cases where you don't want to grant root access to the Yggdrasil service, as such access is required to operate the network configuration during the startup of a new interface.

According to the README:

Yggstack fills the gap by providing SOCKS5 proxy server and TCP port forwarder functionality similar to TOR router. It also can serve as a standalone network node to connect network segments.

How to use

First, run yggstack with the following arguments (replace the values with your own):

./yggstack -useconffile path/to/yggdrasil.conf -socks 127.0.0.1:1080

Now start twisterd with at least the following setup:

./twisterd -proxy=127.0.0.1:1080 -socks=5 -onlynet=IPv6

• the -socks=5 argument is not necessary, as version 5 should be the default (just make sure it is)
• it is important to run connection with -onlynet=IPv6 because yggstack operates with Yggdrasil's IPv6 addresses, while twisterd expects all networks by default (see #16 and onlynet argument usage for details)

Bind on given address

The bind argument may be useful in cases where you have more than one IPv4 or IPv6 interface, such as when using Yggdrasil alongside an Internet IPv6 connection, or when Yggdrasil is configured with a subnet mask (e.g., [300:17a8:aabf:108f::33]), and you want to use only that specific interface to launch twisterd and listen connections on it.

This is also relevant if you are running multiple twisterd nodes on the same host and want to bind a static address for a specific RPC API configuration.

./twisterd -bind=[HOST]:PORT

• to get your current Yggdrasil [HOST], run: sudo yggdrasilctl getself

Firewall examples

Some tips for tuning your iptables

Restrict 0200::/7 range

ufw allow from 0200::/7 to any port 28333

• where 28333 is the port you want to allow for Yggdrasil peers only

Public peers

The twister public peers displayed on the Network page are manually managed to show only those that are actually online.

The following list includes all known peers from that source. Since the Yggdrasil IP address is generated by its owner's private key, it will never expire, and these addresses may come online again at any time. This could be useful when all Internet peers are offline for some reason.

Related software

Some other Yggdrasil-based software you might want to try:

• yggstack - access Yggdrasil through SOCKS proxy without exposing your interface to the network;
• yggmail - end-to-end encrypted email for the mesh networking age;
• Alfis DNS - a peer-to-peer alternative to centralized DNS providers, with native Yggdrasil support, that is also useful for launching a free, independent DNS seeder for the twister network.

Become a public peer

Assist newcomers in discovering other peers by open following ports:

• 28333 - TCP
• 29333 - TCP + UDP
• 4433 - SSL (optional)
• 4434 - SSL (optional)

sudo ufw allow PORT_NUMBER

Mine blocks

Mine blocks and promote your services! This operation aims to store network meta-information on the blockchain.

twisterad

Lightweight, in-memory CLI tool & daemon to rotate multiple twister ads on single worker, through modified Bitcoin Core JSON-RPC API library.

Use cases

• optimal to run as the systemd unit that waits for twisterd connection and then begins updating promotional messages with each new block found
• supports multiple options, including --latency delay in seconds, which is useful for limiting block generation on powerful machines
• written in Rust to ensure memory safety over the long term run, it does not require a disk-based database, making it SSD-friendly for everyday use on your Raspberry Pi!

How does it work

• after launch, twisterad listens for the twisterd connection to be established, and then begins rotation according to the configuration and startup arguments
• when twisterd connection is lost, twisterad will wait for reconnection and continue rotation from the previous memory state. It could be also useful for the desktop users, who running their twisterd nodes periodically

Dependencies

To continue with the cargo command below, follow rustup instructions

Stable

cargo install twisterad

• run twisterad -c path/to/config.json

Repository

• git clone https://github.com/twisterarmy/twisterad.git
• cd twisterad
• cargo run -- -c path/to/config.json

Rotation settings

Check out config.json to setup remote or local twisterd connection and update default promotions asset!

CLI arguments

Usage: twisterad [OPTIONS] --config <CONFIG>

Options:
  -c, --config <CONFIG>        Configuration file, required
  -d, --delay <DELAY>          Rotation queue delay, seconds [default: 60]
      --host <HOST>            RPC host [default: 127.0.0.1]
  -j, --jobs <JOBS>            Processor jobs to mine at once
  -l, --latency <LATENCY>      Miner latency in seconds, useful when blocks are being generated too quickly
  -m, --mode <MODE>            Rotation mode: * `c` - cycle * `s` - stop, disable worker [default: c]
  -p, --password <PASSWORD>    RPC password [default: pwd]
      --port <PORT>            RPC port [default: 28332]
  -r, --rotations <ROTATIONS>  Rotations quantity, before apply rotation `mode`
  -s, --scheme <SCHEME>        RPC scheme [default: http]
  -u, --user <USER>            RPC user [default: user]
  -w, --wait <WAIT>            Wait to server reconnect, seconds [default: 900]
  -h, --help                   Print help
  -V, --version                Print version

systemd

To run twisterad as the systemd unit (background process):

• cd twisterad - navigate sources directory
• cargo build --release - compile optimized binary
• useradd twisterad - create new user for twisterad process
• cp target/release/twisterad /usr/bin/twisterad - copy binary into native system location
• chmod 0700 /usr/bin/twisterad - give required permissions
• chown twisterad:twisterad /usr/bin/twisterad - allow user/group access
• mkdir /var/log/twisterad - create destination for the logs
• cp config.json /etc/twisterad.conf - copy and customize default config

Create new systemd configuration file: nano /etc/systemd/system/twisterad.service

[Unit]
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=twisterad
Group=twisterad
ExecStart=/usr/bin/twisterad -c /etc/twisterad.conf
StandardOutput=file:/var/log/twisterad/debug.log
StandardError=file:/var/log/twisterad/error.log

[Install]
WantedBy=multi-user.target

• to disable debug output, set null for StandardOutput or StandardError

Apply changes:

• systemctl daemon-reload - reload unit configuration
• systemctl enable twisterad - start on system boot
• systemctl start twisterad - launch
• systemctl status twisterad - check service status

On change config.json:

systemctl restart twisterad

Mining with CPU

setgenerate [genproclimit] or use web interface

#Mining with cgminer (solo)

(If you figure this out, please post here)

#Mining with CudaMiner (solo)

CudaMiner-twister now available: https://github.com/miguelfreitas/CudaMiner-twister

Tested with: ./cudaminer -D -o http://127.0.0.1:28332/ -O user:pwd

Make sure your twister-core is up-to-date!

Overview

The twister peer-to-peer network requires a couple of special DNS seeders that are hardcoded in twister-core and twister-seeder to provide an updated list of known peers to the new connections. If you have a 24/7 Linux machine and can add a special NS record to your domain, please consider running twister-seeder!

Requirements

The twister-seeder is based on the bitcoin-seeder so it has similar build requirements

Server

• root access;
• free port 53 to launch twister-seeder DNS on it;
• domain access with the ability to change NS records;
• free centralized domain providers are not recommended, as they may block NS features.

Client (public peer)

To be reachable for twister-seeder, the client (twister-core peer) must meet the following requirements:

• port 28333 is open (twister-core is launched with ./twisterd -port=28333)
• todo: other conditions require a revision of the existing crawl filter implementation

At this moment, the twister-seeder source code is available from two providers:

@miguelfreitas branch

Original version by @miguelfreitas.

Source code

@twisterarmy branch

• contains additional seeds to improve initial crawl connectivity;
• starting from stable version 0.1.0 provides new options:
  • --stats-renew <seconds> - ThreadStats loop delay in seconds (default 1)
  • --no-stats - disable the ThreadStats feature

Source code | Stable releases

Binaries

Some pre-compiled binaries available at the Releases page

Build from source

Debian / Ubuntu

Install required dependencies:

sudo apt install build-essential libboost-all-dev libssl-dev

Log in as root and ensure you are in the /root directory, then follow steps:

1. git clone https://github.com/twisterarmy/twister-seeder.git
2. cd twister-seeder
3. make

Compiled dnsseed binary will be placed in the current twister-seeder build directory (see also Storage location)

Upgrade from repository

Steps to upgrade your existing twister-seeder from the repository sources:

1. stop existing dnsseed process to continue (if active, run pgrep dnsseed to find)
   • sudo systemctl stop dnsseed - if you're using systemd service
2. cd twister-seeder - navigate twister-seeder sources directory (which contains old dnsseed binary)
3. git pull - grab latest updates
4. make - build new version
5. start dnsseed process
   • sudo systemctl start dnsseed - if you're using systemd service

Records example

Create at least two subdomains in your provider's control panel:

• ns.domain.org
  • A:SERVER_IP
• seed.domain.org
  • NS:ns.domain.org

If the provider requires two NS records:

• ns1.domain.org
  • A:SERVER_IP
• ns2.domain.org
  • A:SERVER_IP
• seed.domain.org
  • NS:ns1.domain.org
  • NS:ns2.domain.org

Alternative domain providers

Alfis DNS

Alternative Free Identity System - decentralized domain registration in the blockchain;
requires renewal once per year by the PoW (Proof-of-Work).

This solution is not compatible with Clearnet domain zones but supports Clearnet IPs.
It also supports Yggdrasil out of the box.

#/etc/systemd/resolved.conf
DNSStubListener=no

Firewall

Change the iptables rules to make twister-seeder DNS accessible from outside:

sudo ufw allow 53

Launch

Login as root then run:

./dnsseed -h seed.domain.org -n ns.domain.org

Output example:

0/2 available (2 tried in 333s, 0 new, 0 active), 0 banned; 31 DNS requests, 13 db queries
...

Test

Before continue with systemd,
run on the local/remote host following command when twister-seed (dnsseed) is running:

host -a seed.domain.org ns.domain.org

the output appear as follows:

Trying "seed.domain.org"
Using domain server:
Name: ns.domain.org
Address: IP#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18126
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;seed.domain.org.		IN	ANY

;; ANSWER SECTION:
seed.domain.org.	40000	IN	NS	ns.domain.org.
seed.domain.org.	60	IN	A	TWISTER_CORE_IP_1
seed.domain.org.	60	IN	A	TWISTER_CORE_IP_2

Received 99 bytes from IP#53 in 0 ms

• if everything is working, TWISTER_CORE_IP_1 and TWISTER_CORE_IP_2 should display the real peers who are running their twister-core nodes on port 28333
• if you have any issues with discovering new peers, see Troubleshooting or open new Issue

systemd

# /etc/systemd/system/twister-seeder.service
[Unit]
After=network.target

[Service]
Type=simple
ExecStart=/path-to/twister-seeder/dnsseed -h seed.domain.org -n ns1.domain.org
# please note:
# twister-seeder may generate massive output if the StandardOutput is set to a log file!
StandardOutput=null
StandardError=file:/path-to/twister-seeder-errors.log
Restart=on-failure

[Install]
WantedBy=multi-user.target

• systemctl daemon-reload - apply systemd update
• systemctl enable twister-seeder - launch twister-seeder on system startup
• systemctl start twister-seeder - start DNS

When launching the twister-seeder as the root user, current implementation saves data files to the system root (/) location instead of the typically expected /root/* directory:

• /dnsseed.dat
• /dnsseed.dump
• /dnsstats.log*

By using systemd to launch the service, log files will be available in the locations defined in the StandardOutput and StandardError settings

To make new twister-seeder visible for other peers, create new PR to following lines:

• twister-core (example)
• twister-seeder (example)
• optionally, submit new PR to the Network page, which is displayed on the community website (example)

DNS can't discover new peers

DNS cache

First of all, check that the DNS cache has really been updated; it may take some time for the domain settings to change.

Satisfy public peer requirements

Make sure that at least one expected twister-core instance (with open port 28333) has been online for about one hour or more. This peer should also be synchronized with the network, meaning it should contain the actual blockchain version and have established DHT connection.

Emit a signal from new peer

Try sending a new request to your new twister-seeder from the peering twister-core, as your twister-core probably doesn't know about the new twister-seeder since it's not listed and can't obtain its address from other peers: open the web UI and add your seed.domain.org to the DNS to obtain list of peers entry.

Clear the crawl cache

Current twister-seeder implementation may temporarily ban unstable public peers. This behavior is inherited from the legacy bitcoin-seeder, which filters out public peers that are not running on port 28333 or those with low connectivity.

To reset twister-seeder crawl cache, run ./dnsseed with following options:

• --wipeban - wipe list of banned nodes
• --wipeignore - wipe list of ignored nodes

Manually define a new peer

Finally, if the twister-seeder outputs 0/2 available for a while, and you already running or know an existing twister-core IP with active connections (on port 28333), try defining it as the initial peer by temporarily hardcoding the following line in main.cpp, then recompile and restart dnsseed:

CDnsSeedOpts *opts = (CDnsSeedOpts *)arg;

add after:

db.Add(CService("NODE_IP", 28333, true), true);

• where NODE_IP is public twister-core that running on port 28333

Domain provider disallowed changes to NS record

Free domain providers (like DeSEC) may drop NS as the abusive feature.

That is the reason why free domain providers are not recommended for twister-seeder, as old twister-core peers are already using this address for connectivity checks. As a solution, if you run a public twister-core with a static IP (or know stable peers with static IPs), try adding their IPs manually as A records for the nameserver:

• ns.domain.org
  • A:TWISTER_CORE_IP_1
  • A:TWISTER_CORE_IP_2
  • ...

The twister-rss-bot is RSS bot for twister P2P written in PHP 8.

It uses the twister-php library to interact with twisterd node through the JSON-RPC API and supports following features:

• multiple feed providers
• custom tags configuration
• can send messages to different twister accounts (that you own)

Requirements

Installation

apt install composer git

Runtime

apt install php-curl php-fpm php-mbstring php-pdo php-sqlite3 php-xml

Install

• git clone https://github.com/twisterarmy/twister-rss-bot-php.git
• cd twister-rss-bot-php
• composer update

Setup

• cp config.example.json config.json
• nano config.json - setup your twister connection, tags and RSS feeds!

Usage

• crontab -e - open crontab configuration from system user
  • @hourly php src/cli/bot.php - update feeds every hour (see more examples)

Common

• @echotest Echo [Echo test for your twists (up to 5 minutes delay, please have patience)]
• @tiktok Tiktok [UTC Hourly heartbeat]
• @potm Phase of the Moon

English

• @twister twister [twister announcements. low volume.]
• @mfreitas Miguel Freitas [hacking twister in spare time]
• @letstwist Let's Twist! [Git commits for twister-core and twister-html]
• @newusers New users [Announcing new users as they register in the blockchain]
• @blockhash Block Hash [Counting the blocks in the Twister blockchain]
• @userstats Twister User Stats [Some infos about twister users, http://yazgi.net/twister/users]
• @calm_client Calm [Calm is an alternative client for twister, https://github.com/iHedgehog/twister-calm]
• @theme_nin Nin Theme [Twister Theme Nin, https://github.com/myleneb/twister-nin]
• @hn Hacker News [https://news.ycombinator.com/]
• @bbc_world BBC World News [BBC World News is the BBC's international news and current affairs television channel, http://www.bbc.com/news/world/]
• @intercept The // Intercept [Fearless, adversarial journalism, https://firstlook.org/theintercept]
• @colossal Colossal [Art, design, and visual culture, http://www.thisiscolossal.com/]
• @rt_com [Russian propaganda TV network, http://rt.com/]
• @lgbt LGBT [Lesbian, gay, bisexual, and transgender]
• @soltempore Sol Tempore [https://twitter.com/soltempore]
• @fourtwenty 16:20 [There's one every hour somewhere, http://is.gd/fourtwenty]
• @torrentfreak_com TorrentFreak [Breaking File-sharing, Copyright and Privacy News, http://torrentfreak.com/]
• @schneier_sec Schneier on Security [A relay from Schneier's blog, https://www.schneier.com/]
• @smashingmag Smashing Magazine [Smashing Magazine is an online magazine for professional Web designers and developers, with a focus on useful techniques, best practices and valuable resources, http://smashingmagazine.com/]
• @manliness The Art of Manliness [Reviving the lost art of manliness since 2008, http://www.artofmanliness.com/]
• @meduza_en Meduza.io #en [The Real Russia, Today, https://meduza.io/en]

French

• @lemonde_rss Le Monde [Je vous apporte le RSS du Monde.fr sur un plateau d'argent!, http://lemonde.fr/]

Russian

• @rutwister RU - Twister [Новости Twister'a на русском]
• @rublacklist РосКомСвобода [http://rublacklist.net/]
• @i2pnews I2P news [Новости из мира I2P]
• @habr_ru Habrahabr.ru [http://habrahabr.ru/]
• @geektimes GeekTimes [http://geektimes.ru/]
• @piratemedia Pirate MEDIA [Информационно-новостной портал Пиратской партии России, http://piratemedia.net/]
• @slon_ru Slon.ru [Деловые новости и блоги, http://slon.ru/]
• @theins_ru The Insider [Журнал о расследованиях, http://theins.ru/]
• @bbc_russian BBC Russian [Русская служба BBC, http://www.bbc.co.uk/russian/]
• @sputnikipogrom Sputnik & Pogrom [Спутник и погром, http://sputnikipogrom.com/]
• @b0ltai Shaltay Boltay - Ретранслятор [http://b0ltai.org/]
• @petrimazepa Пётр и Мазепа - Ретранслятор [http://petrimazepa.com]
• @apparat_cc Apparat [Журнал о новом обществе, http://apparat.cc/]
• @meduza_io Meduza [Meduza — новости возвращаются, https://www.meduza.io/]
• @tvrain Телеканал ДОЖДЬ [Все новости телеканала ДОЖДЬ, http://tvrain.ru/]
• @besttoday_ru BesTToday [Besttoday.ru - всё самое лучшее в блогах, http://besttoday.ru/]

Chinese

• @bbc_zhongwen BBC Chinese [新闻主页, http://bbcchinese.com/]
• @reuterscn Reuters 路透中文 [http://cn.reuters.com/]
• @voa_cn 美国之音中文网 [http://voachinese.com/]
• @newscn 喧哗社 [信息的自由流动是辟谣的最佳方式]
• @freeweibo 自由微博 [最新最热微博(包括屏蔽的)机器人。数据由自由微博http://Freeweibo.com 提供。]

twister 白皮书 （中文翻译）

twister - 一个P2P的微博平台

Miguel Freitas

摘要

这篇白皮书提出了一种新的基于P2P网络覆盖层的微博架构。该平台由三种完全独立的网络覆盖层组成。 第一个网络覆盖层提供分布式的用户注册和验证，基于比特币协议。第二个是分布式的哈希表网络覆盖层，用 来存储第三个网络覆盖层中用到的用户资源和tracker位置的键值。最后一层网络层是粉丝swarms的可能的解体， 基于BT协议，这个可以用来实现向多用户有效地发送近点通知。通过融合一些已经存在和验证过的技 术，twister提供了一种新的微博平台，能够提供安全，可伸缩和私密的特性。并有一种机制能给贡献处理时间来 运行用户注册的网络实体以激励，奖励它们向整个网络发送一条推送（“”广告”）消息的特权。一天推送消息 的数量以不使用户厌烦为基准。

1 介绍

微博平台是当今互联网上最多元和最有影响力的技术之一。最近的事件显示了这些工具对新闻传播以及政治 运动，如中东的‘阿拉伯之春’运动的重要作用。尽管它们在社会革命中的作用不应该被过分夸大，但这是一个示范 可以让我们去了解为什么独裁政权经常试图关闭互联网以控制这些可能的颠覆运动。然而阻止对互联网的访问，从 来不能完全有效地抵抗社会运动，因为人们总是能找到途径突破这些封锁。

社交服务提供商可能会乐意参与对社交媒体的封锁，相对于仅仅封锁一些网络主干“backbones”，这种可能性更 加可能影响人们以更生动的方式进行沟通的能力。随着我们社会对这些服务依赖的增加，这些基本的交流平台的 任何一点失败，不仅是不能接受的，更是直接违背了互联网设计的关键特性：为信息传输提供冗余。

有关大公司配合的大范围的互联网监听活动的报告，揭示出这些平台侵犯用户隐私的危险。这种一个实体只要他们 愿意就可以访问用户私人交流和个人信息事实应该引起任何的人的担心，只要他认真想一想的话。 一份当前的 House of Lords（UK)的报告公开指出这种大规模监听的危险。

所有这些因素指出了一个明显的方向：现在迫切需要开放，安全和分布式的个人交流平台。这就是当前这个P2P 微博平台提议适合的地方。 当然，要想成功，这些P2P微博平台不能仅提供灵活性和安全性，它还必须对用户友好。这是发展任何新的 软件和网络服务的关键。 当前一些P2P信息提议给这种不知道怎么做用户友好提供了例子，例如要求用户记住一个 模糊的36位大小写敏感的字母组成的地址。 能提供用户一个容易记忆的用户登录必须被当作一个关键需求考虑。当用户必须自由的选择他们的登录名，给那些 想要自由表达而不用担心被报复，认识到这点很重要，在这些微博架构上建立一个可信任的网络取决于这些真实 存在且完全可辨认的人。这个问题可以在Hudson坠机新闻中体会出来，那里受信任的aggregator帮助将可靠的 信息从随机的噪声中分离出来。这些在信息传输中像hub一样工作的人，经常被定义为”influential”。 任何严肃的P2P微博提议都必须培育这种类型的组织。

这份白皮书阐述了一个新的P2P微博平台提议，一个关于可扩展，有灵活性的面对失败和攻击，不需要依赖任何中心认 证中心来进行用户注册，提供易于使用的加密了的私密交流和认证过的公开帖子。该架构试图尽可能融合几种已有 的且验证过的P2P技术，比如BT和比特币。私密性同样是主要的设计考量，应该没有人能看到用户的IP或他们粉丝 ，除非他们明确分享这些信息。

提议的平台由三个非常独立的覆盖层网络组成。第一个基于比特币协议，提供分布式的用户注册和认证功能。 第二个是一个分布式的哈希表覆盖层网络，用于第三个网络中用到的用户资源和tracker位置的键/值存储。最后 一个基于BT协议，是粉丝“swarms”的可能的分离的集合，可以被用来进行有效的多用户近点通知的发送。

2 相关工作

现有的社交网络如Daspora，StatusNet和identi.ca，经常作为Facebook和Twitter的分布式的，自由的替 代品被提及。 这些平台是基于”联合社交网站“的概念，用户可以选择加入这些社交网站，而这些网站通过开放的 协议彼此之间相互通讯。当为了获得更好的隐私控制而专门地选择某个单一的，封闭的平台，用户还是要把他们 自己的数据交给第三方（除非他想设置他自己的服务器来联合）。

以前也有P2P的微博平台提案，像Cucko[12]和Megaphone[13].Cucko和Megaphone看起来都不是为了解决 分布式用户注册的问题。隐私同样不是Cucko的目标之一，因为它直言设计用来了解任何用户的线上状态的。 twister和Cucko的一个共同点是都使用一个非结构的覆盖层网络来进行用户帖子的传播的思路，不像Megaphone 所有粉丝必须注册到Sender，形成一个多点传输树来进行帖子的传播。

一个更高级的社交网络提议Safebook瞄准好几个隐私问题，通过实现不同的层级（”shells“）来访问发布的 数据。Safebook的目标范围比twister大得多，它同样依赖一个中心化的受信任的识别服务用于用户的注册。

目前为止，这些P2P提议似乎都没有公开的实现。

3 标注

元组：[a, b, c, ...]

应用函数f到 payload x:y = f(x)

用户j的地址： IDj = H(Username_j); 其中H是个hash函数

用户j的公开密钥和私人密钥： PUBK_j; PRIVK_j

备注：PUBK（PRIVK（x））=x, PRIVK(PUBK(y))=y

用户j的签名内容x：SIGj(x)=[PRIVK_j(H(x)),x]

4 P2P网络的用户注册

去中心化的且安全的用户注册功能已经通过块链机制得以实现，这种机制被比特币用来在不需要中心认证的情况下， 解决”重复发送“问题。提议中的系统这个机制用来保证用户的唯一性，同样不需要中心认证。新的注册在被认为 确认是一个特定的用户前，必须被一系列的块“notarized”。每一个块被这样定义：

Block_i = [i, H(Block_i-1),Nonce_i, SpamMsg_i,[UserReg_j, UserReg_j+1,...]]

H（Block_i）凭借部分哈希碰撞（通过野蛮地强制在Nonce_i空间上搜索），产生工作证（POW）。困难在于网络 如何基于每小时块产生的平均数来自动设置（和比特币一样）。 UserReg_j = [Username_j, PUBK_j, Nonce_j]

当一个新用户注册到网络，在他的请求被重发和接受之前，他必须广播UserRegj。其他节点，根据收到的UserReg_j 必须通过一个部分哈希碰撞H(UserReg_j）来检查POW。这个工作证可阻止DoS攻击，因为通过伪造注册的泛洪flooding。

UserReg_j的POW比块链的POW小得多，通常只是几分钟的平均电脑时间（可能可能是写进代码并仅仅随着协议版 本改变）。

块链提供了公开的从Username_j（或ID_j）到PUBK_j的字典。

在将UserRegj包含进一个新块之前，节点必须强制执行Username_j的唯一性。该规则唯一的例外是密码替换的 情况，新的公开密钥由以前已知的密钥对签名。ID_j和UsrReg_j的POW的唯一性的强制检查也应用在接收新块的 时候，因为所有的注册包括其自身都必须被检查。

Username_j同样受一些额外文本规则的限制，比如最大长度和允许的字符。这些更好地保护了ID空间免受部分哈希 碰撞攻击，相反的，也意味着容易被监视（监听，见章节12）。

SpamMsg_i是一个必须被所有客户端显示的非请求消息（通常委婉地称做推广信息），提供了加入块链产生运动 的一个激励。如果一比特币块产生的相同速率（每小时6块），显示的一个可能性因子可能按不使用户看到太多广 告而厌烦为标准。

开发者千万不能开发隐藏广告信息的功能来作为他们的客户端特性，因为这个奖励对整个网络的安全起到重要作用。 从客户端软件中删除非请求信息只会伤害用户。显示可能性因子可以被实现，同样，客户端软件可以调整本地 化的优先级（通过给和用户同样语言的信息更高的可能性）来提高效率和用户体验。

5 可路由的DHT网络覆盖层

第二个P2P网络是一个结构化的分布式哈希表覆盖层网络，类似Kademlia[16]。这个网络唯一最重要的功能 是允许peer存储和获取资源。直接投递用户间提醒可以被算作第二个用途（见章节7）。

直接使用IDj作为端的地址加入DHT网络是冒险的，因为这会允许简单的条件反射认证，可能会阻止ID伪造。在 P2P/DHT网络中，伪造的ID地址是毋庸置疑的最严重的安全问题（参考Sybil和Eclipse攻击[17][18]）。 使用ID_j作为DHT的地址会极大地危害隐私，因为这种网络的一个根本特性就是通过了解其他节点的IDs来建立优化路由表的。 ID_j不仅能轻易侦测用户的在线活动，而且还可能暴露用户的IP地址。 相对于ID_j，提案中使用用IP地址和端口号进行哈希运算的标准进程来加入DHT网络： ID_{node_j} = H（[IP_j，port]）。

在[19]中，显示了限制到每次活动中，从外部IP到ID的安全映射是防攻击的。 这个DHT网络中从IDsrc发送到IDdst的数据包，其定义如下： Package = [ID_dst, ID_src, SIG_j(payload), ID_j)] payload由给定的用会ID_j签名，甚至在信息被转发或刷新的情况下，它都可能和发送者ID_src不一样。

这些由这个覆盖层网络所提供的特性组成了“第三层”的基本功能。 在提议的DHT覆盖层网络概念模型中，向上走有一层通过数据存储术语（PUT）操作的“应用层”，，用下面的负载 粗略定义：

payload_PUT = [target, value, time, seq] ， 其中target = [owner, resource, restype] ，ID_dst = H（target）

为了接受存储请求，目标节点必须检查一些简单的规则：

1. ID_dst = H（target）：确保目标地址是被合理地计算生成的。
2. ID_dst 是那个实际接收请求的ID_node的邻居。（通过一些共同同意的测定方法）
3. ID_j = H（owner），仅仅强制 restype = "single"
4. seq比之前储存的seqold大，仅仅强制要求 restype = "single"
5. 时间是一个有效的时间（比如，不能是未来）

restype两种可能的值是”single“和”multi”。这两种类型相应地提供，仅仅可以被该密钥所有者更新的内容 （如用户头像），或集合了不同用户的多条回应的资源（如对特定帖子的回复）。在“sigle”的情况下，节点仅仅 保存公开密钥IDdst相对应的一个内容。而在”multi“状况下，新的PUT请求被附加到一系列的内容上。这种类 型的存储不提供任何的保证，内容可能会根据失效条件（基于时间field）或”最小当前使用（LRU）”缓存策略而 被舍弃。认证过的（“single”）存储优先于任何前面的”multi”内容。

数据检索操作符（GET）可以不加区分地操作这两种类型的资源。一些特殊的与动态内容相关的非存储资源也 可以使用相同的操作符来进行操作，同样共享相同的API。

6 用户帖子

用户j的第k个帖子被定义如下：

UserPost_jk = SIG_j([Username_j, k, type, MSG_k, REPLY_k])

其中MSGk是帖子的内容（限制最多140字符），k是一个递增的数字， type可以定义这是一个新帖子，或是 回复，或是转发（RT），或者私信（DM）。REPLYk是一个可选的字段，提供了一个到原始帖子的指针引用，在回 复或转发（RT）（参考章节8）情况下，定义为一个元组REPLY_k =[Username_j',k']，其中原始帖子是用户 j‘的第k’条帖子。

帖子在两层网络覆盖层中同步共享：（1）作为DHT网络中的存储内容，可能只存在较短时间；（2）从属于某种 BT网络的类似文件的存档。当创建一个新帖子的时候，客户端软件必须发送两个PUT请求到以下地址：

ID_{UserPost_jk} = H ([Username_j,"post"+k,"single"])， 和

ID_{swarm_j} = H ([Username_j,"swarm","single"])。

ID_{UserPost_jk}是一个存储目标的地址，在章节5中有定义，同时提供任意帖子的获取能力。

ID_{swarm_j}是一个能访问种子文件 swarm的特别的网关地址（在BT术语中[20]）。这些种子文件可能包含 指定用户j的所有帖子，并协助在DHT网络中独立地共享它们。IDswarm_j的邻居被要求加入这个swarm， 尽可能多的ID_{UserPost_jk}的邻居被要求存储这些内容。DHT-种子文件 的交互规则在章节7.2中有进一步的 详细说明。

新帖分发的swarm机制修复了新帖提醒的效率问题，避免了粉丝们轮询一个DHT网络中特定地址的需要。这是一个 有同样问题（“lame，重复轮询”）的不同解决方案，该问题由pubsubhubbub协议后面的开发者引起。

6.1 私信

如果接收者是用户k的粉丝的话（和Twitter的要求一样），用户帖子同样可被用来发送私信（DM）。

UserPost（j->l)k, = SIG_j(["",k,"dm",[PUBKl(DM_k),H(DM_k)]])

大家应该注意到DM与一个普通的帖子是一样的，除了用[PUBK_l（DM_k),H(DM_k)] 替代了上面一般公开帖子中的负载。 通过检查解密是否成功，DM只会被目标用户l接收到。没有其他用户会知道DM的接收者是谁，尽管加密过的信息 会被所有他的粉丝看到。

这个关于DM加密机制的简陋说明仅仅是想用来解释这个概念，而实际的实现可能会不同。当前，这个工作的twister 原型基于ECIS（Elliptic Curve Integrated Encryption Scheme），是由Ladar Levison[22] （Lavabit加密邮箱服务以前的拥有者）开发的，推测会符合 SECG SEC1标准。

6.2 用户帖子 种子文件/tracker规则

• ID_{swarm_j}哈希空间中特定距离内的在线邻居，被要求加入（或创建）swarm。
• 当某个ID_{swarm_j}的邻居从DHT网络中接收到一个新帖子，他必须像网关一样工作，把帖子合并成一个可在BT 网络中分享的类似文件的结构。
• BT tracker是一个特殊的可读的多值列表存储，可通过公式 ID_{tracker_j}=H([Username_j, "tracker","multi"])寻址。
• 用户j的粉丝必须加入swarm来接收实时的更新。他们通过查询(GET操作)ID_trackj来获得一个 初始点的列表的方式来完成这个工作。
• ID_{track_j}与其他存储键不同的是它的只读属性。这是一个安全措施用来防止tracker污染，同时用来保护 swarm成员的隐私。因此IP地址列表是从swarm协议自身（BT）得到的，而不是从DHT网络以可读写的方式产生。 这增加了一个额外的要求：ID_{track_j}的在线邻居也被要求加入swarm。
• swarm成员之间只能通过IP地址了解其他成员。这个类似BT的网络务必不提供任何有关用户名的线索。
• 这里不需要一个关于所有帖子的哈希表（比如，种子文件的块校验），因为所有帖子（包括私信）已经被 签名并且可以被验证。
• k值的增加（新帖子）是由swarm内部的广播直接传播的（flooding）。
• swarm成员相互之间交换可见帖子的位列表。成员可以选择仅保存/请求最近发的帖子。
• Seeders是志愿成为档案保管员的节点。
• 发帖者（用户j）可以选择不作为自己swarm的成员（为了隐私目的，保护他的IP）。
• 如果发帖者选择成为自己swarm的成员，他可能跳过了整个ID_{swarm_j}网关的算法，会损失一些IP隐私。
• 即使发帖者是自己swarm的成员，他也不需要成为seeder。
• 在BT术语中，存在的pieces的数量必须通过新帖子增加到k。这通过发送一个（非请求）“有”的消息来实现。
• 客户端必须把”有“消息的参数当作新pieces的数量来对待。为了防止DOS攻击，这个数值必须限制在 k < 2*(iBlock_current - iBlock_{User_reg})+20. 否则数字会被替换。
• 如果一个新block k每10分钟产生一次，这个限制意味着新用户的发帖率最大是288帖每天。平均来说。

7 提及（@）

如果一个新帖子（@username)提及了用户j，客户端软件也必须发一条提醒到ID_j，通过包含完整信 息的方式。提醒通过DHT网络路由过去。 提及是提案架构中唯一需要路由数据包到指定用户地址ID_j而不是IDnode_j的特性。另一种选择是，一个不同的

ID_{mention_j} = H([Username_j, "mention"])

可以被设置用来接收和累计提及，由ID_{mention_j}的邻居节点维护。这里唯一的问题还是“lame，重复pulling”， 因为用户需要定期地提取这个密钥（尽管比起hashtag来范围更受限）。 一个可以阻止提取用户提醒而且能一定程度保护用户隐私的方式是为目标ID_j选择一些”收听者“。然后这些listeners 可以将数据包抄送到最终的用户。这些思路部分是基于SASON，尽管不是太安全，因为没有使用额外的匿名网络。

系统可能像这样工作：接收者IDj首先使用DHT网络找到ID_j旁边的node。 然后他要求他们直接抄送所有ID_j的 流量到ID_{node_j},因此只暴露他的真实身份给一小组他的lisoners。Listeners必须通过询问SIG_j（随机号码） 做一个challenge验证，以确保用户是真正的ID_j。因为其他节点已经访问到完整的公开密钥的字典，所以他很 容易认证。

提及，像这里描述的其他机制一样，需要客户端软件的配合才能工作。如果指定的用户不发送提醒包到网络中（和 他自己的帖子一起），那被提及的用户就永远不会知道。

8 显式信息请求

用户l不用加入swarm，就可以显式地向用户j请求某个特定信息。这通过从地址IDuserpostjk获取的一个简 单的认证内容来完成。 该特性能够像Twitter一样进行“上行消息分支导航”，并且不需要太多资源。

9 下行信息分支导航

下行导航（找出特定帖子的所有回复和转发）可能是一个困难的问题，因为”这个帖子的回复是什么？”的问题， 可能有许多，甚至可能无限的答案。一个可能的方案是另外发送一个提醒到一个特定的多值列表存储地址： ID_{replies_jk} = H([Username_j,"replies"+k, "multi"]) 要存储的内容是回复本身的复制（用户帖子的格式在章节6中定义）。同样，要使这个机制工作，需要客户端合配合 发送回复。

10 哈希标签

像提及一样，hashtags必须在发布到网络中的新帖子的内容中被检测到。一份复制的信息被发送到一个特殊的 多值列表存储的地址： ID_hashtagt = H([hashtag_t,"hashtag", "multi"]) 这和下行信息分支的导航非常相似，除了一个额外的特性：hashtag创建了一个与ID_{swarm_j}相类似的新swarm。 这个ID_hashtagt的邻居被强制加入这个虚拟的没有连续内容（文件）的swarm。含有hashtag的帖子被DHT 路由到一个swarm的邻居成员，从那里它们被广播到swarm的成员。

因此swarm仅被用来创建一个分布式的tracker和广播机制，给愿意监控这些hashtags的用户。新加入swarm 的成员也可以从多值存储中（DHT网络）请求最后的信息，但不保证完整性。

11 文字搜索

任意单词的搜索，可以将哈希标签的实现思路扩展到所有帖子中的所有单词中。为了减少开销和网络流量，可以 进行特定的限制，如最小单词大小，去除介词等等。 和哈希标签的另外一个区别是，为所有可能单词创建swarms被认为是过度杀伤的。所以包含某个给定单词的 帖子的集合应该被限制在一个临时的多值的列表存储中，通过下面公式寻址： ID_wordw = H([word_w,“word”, “multi”])

12 最后备注

提议的架构提供了一个分布式的P2P的微博网络，拥有安全，可伸缩和私密的特性。

• 架构是像其他的P2P技术一样是有灵活性的，所以被认为没有单个的公司，政府或其他的实体能够关闭它。
• 分布式的用户注册机制像比特币交易一样安全，能提供内容认证而不依赖任何特殊的实体。
• 真实用户为了注册他们钟意的用户名，有动力在早期就使用这个平台。
• 使用常用的用户名，而不像在其他一些平台一样使用很长的加密哈希值，可以使系统像现在的微博系统一样对 用户友好。
• 当用户的安全性受到任何方式的危害时（比如手机被偷），公开密钥替换功能允许用户改变他的密钥对。这同时 也使得用户或公司购买他们的用户名成为可能（比如已经存在的商业域名）。
• 现有其他微博系统的主要功能都已经实现，包括搜索用户名，“导航跳转，提及他人”，私信，标签和文字搜索。
• DHT路由功能提供了一条途径，可以向特定客户发提醒或向他请求资源数据（头像，用户资料等），而不需要 知道他是否在线。
• 为了检测一个用户的IP地址或监听他的活动，该实体可能需要试图分配自己一个靠近受害者（或者一个它 所属的资源，如tracker）的IDnode。因为有从外部IP地址对IDnode认证的限制，这不是一个容易的工作。
• 拥有巨大资源的组织（有众多可选的IP地址区块）可能能够完成这个部分ID_node碰撞，以侦查一个特定用户 的活动。这使得网络监听能力从大规模的监听转移到相对合理的有目标监听。
• 尽管发现一个用户的在线活动是困难的，但这不是这个架构的严格保证。要求更高私密性的用户，建议在tor 网络的基础上使用twister。
• 为了获得有限频率的广告发布权利，该架构为公司运行该系统提供了动机。这可以是盈利性的广告， 但它同时也允许一群用户加入一个社区组织来传播一些信息（如比特币的挖矿池）。所以该提议同样相当民主。
• 独立提供商们可以使用一个标准的网页界面提供来twister访问，而在后端加入P2P网络。然而，尽管这完全 合法且完全支持，但这种模式破坏了大部分的隐私和安全特性，因为提供商拥有用户的私人密钥（PRIVK)。
• 一个聪明的网页界面的解决方法可能是存储一个密码加密过的私人密钥（PRIVK）版本在服务器上，那样为了 发一条新消息，私人密钥被运行在浏览器中的javascript暂时解密。这个办法可以阻止服务器的拥有者拥有用户 的私人密钥。
• 一个用于读取用户的公开帖子和标签的只读网页界面是可行的，而且也不会危害安全。
• 资源有限的客户端（如手机）可以选择做一些优化。比如，他们可以不存储完整的块链，而只存储区块哈希表 的链。为了搜索一个特定的用户，他们可能要询问网络哪些些区块明确包含该用户的注册信息。然后客户端软件可 以只下载需要的区块而不损失任何的安全性（块的完整性是可验证的）。相对于下载整个区块，可以使用一个部分 Merkel树。

参考

[1] Rachel Sklar. Hudson plane crash on twitter: First reports,best coverage. MEDIAite http://www.mediaite.com/online/ hudson-plane-crash-on-twitter-first-reports-best-co verage/,2009. [Online; accessed 23-July-2013].

[2] Habibul Haque Khondker. Role of the new media in the arab spring. Glob-alizations, 8(5):675–679, 2011

[3] James Glanz. How mubarak shut down egypt’s inter-net. The Age World http://www.theage.com.au/world/ how-mubarak-shut-down-egypts-internet-20110216-1awjj.html, 2011. [Online; accessed 23-July-2013].

[4] Margaret Warner. Syria internet shutdown: A loser’s strat- egy. PBS Newshour http://www.pbs.org/newshour/rundown/2012/11/ syria-internet-shutdown---a-losers-strategy.html, 2012. [Online; accessed 23-July-2013].

[5] Adam Dachis. How to foil a nationwide internet shut- down. lifehacker http://lifehacker.com/5746046/ how-to-foil-a-nationwide-internet-shutdown , 2011. [Online;accessed 23-July-2013].

[6] Josh Halliday. Facebook and twitter to oppose calls for social media blocks during riots. The Guardian http://www.guardian.co.uk/media/ 2011/aug/24/uk-riots-facebook-twitter-blackberry, 2011. [Online; accessed 23-July-2013].

[7] Wikipedia. History of the internet. http://en.wikipedia.org/wiki/ History_of_the_Internet#Packet_switching, 2013. [Online; accessed 23-July-2013].

[8] Glenn Greenwald et al. How microsoft handed the nsa access to encrypted messages. The Guardian http://www.guardian.co.uk/world/2013/jul/11/microsoft-nsa-collaboration-user-data , 2013. [Online; accessed 23-July-2013].

[9] House of Lords (UK). Surveillance: Citizens and the state, volume i. http://www.publications.parliament.uk/pa/ld200809/ldselect/ldconst/18/18.pdf , 2009. [Online; accessed 29-July-2013].

[10] Jonathan Warren. Bitmessage: A peer-to-peer message aut hentication and delivery system. 2012.

[11] wikipedia. Distributed social network. http://en.wikipedia.org/wiki/Distributed_social_network , 2013. [Online; accessed 29-July-2013].

[12] Tianyin Xu, Yang Chen, Jin Zhao, and Xiaoming Fu. Cuckoo: towards decentralized, socio-aware online microblogging services and data measurements. In Proceedings of the 2nd ACM International Workshop on Hot Topics in Planet-scale Measurement, page 4. ACM, 2010.

[13] Timothy Perfitt and Burkhard Englert. Megaphone: Fault toler ant, scal-able, and trustworthy p2p microblogging. In Internet and Web Applications and Services (ICIW), 2010 Fifth International Conference on , pages 469–477. IEEE, 2010.

[14] Leucio Antonio Cutillo, Refik Molva, and Thorsten Strufe. Safeb ook: Aprivacy-preserving online social network leveraging on real-life trust. Com-munications Magazine, IEEEs, 47(12):94–101, 2009

[15] Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system. Con-sulted, 1:2012, 2008.

[16] Petar Maymounkov and David Mazieres. Kademlia: A peer-to-pe er infor-mation system based on the xor metric. In Peer-to-Peer Systems , pages 53–65. Springer, 2002.

[17] Lin Wang. Attacks against peer-to-peer networks and count ermeasures. In T-110.5290 Seminar on Network Security, 2006.

[18] Yu Yang and Lan Yang. A survey of peer-to-peer attacks and counter attacks. In International Conference on Security & Management (SAM 2012) , pages 176–182, 2012.

[19] Jochen Dinger and Hannes Hartenstein. Defending the sybil attack in p2p networks: Taxonomy, challenges, and a proposal for self-registration. In Availability, Reliability and Security, 2006. ARES 2006. The First Inter- national Conference on, pages 8–pp. IEEE, 2006.

[20] Wikipedia. Glossary of bit种子文件 terms. http://en.wikipedia.org/wiki/Glossary_of_Bit种子文件_terms , 2013. [Online; accessed 23-July-2013].

[21] Brad Fitzpatrick et al. pubsubhubbub - a simple, open, webhook based pubsub protocol and open source reference implementation. http://code.google.com/p/pubsubhubbub/, 2013. [Online; accessed 24-July-2013].

[22] Ladar Levison. Code for using ecies to protect data (ecc+ aes + sha). http://openssl.6102.n7.nabble.com/ Code-for-using-ECIES-to-protect-data-ECC-AES-SHA-td39269.html , 2010. [Online; accessed 1-October-2013].

[23] Standards for Efcient Cryptography Group. Sec1: Elliptic curve cryptog- raphy, ver. 2. http://www.secg.org/download/aid-780/sec1-v2.pdf ,2009. [Online; accessed 1-October-2013].

[24] Henry Tsai and Aaron Harwood. A scalable anonymous server overlay network. In Advanced Information Networking and Applications, 2006. AINA 2006. 20th International Conference on, volume 1, pages 973–978. IEEE, 2006.

[25] Tor项目. Tor (洋葱头路由).https://www.torproject.org, 2013. [Online; accessed 23-July-2013].