= The Newsboat Hacker's Guide
Andreas Krennmair <ak@newsbeuter.org>
Alexander Batishev <eual.jp@gmail.com>

**WARNING**: this document is being slowly updated. See the date of each
section to get a feel of how up-to-date and reliable is the information in it.

== Introduction

This is the "hacker's guide" to Newsboat. It describes the overall architecture
of the program, the most important design decisions, and some other noteworthy
things that don't belong anywhere else.

This guide assumes that you know about Unix programming, C++, Rust,
multithreading, and some more stuff. This is not for end users.

== Architecture

=== Classes

**Status of this section:** incomplete (January 2021).

This section describes the different classes and their purpose.

*class cache*: the persistence backend of newsbeuter. It takes rss_item and
rss_feed objects and writes them to the cache.db using the SQLite library,
respectively reads the content of cache.db and creates rss_item and rss_feed
objects.

*class colormanager*: manages the color configuration. It hooks into the
configuration parser, stores the color configuration information and sets
them in the view. This is currently a bit ugly, because colormanager knows about
the internals of view resp. pb_view. This should be changed in a way that
colormanager knows nothing about who gets the configuration, and that the views
retrieve this configuration from colormanager. It is derived from
config_action_handler, a helper class for the configuration parser.

*class configcontainer*: manages the normal program configuration. It hooks into
the configuration parser and stores the configuration information. Other
components can then query the configcontainer for configuration values. It is
derived from config_action_handler.

*class configparser*: parses the configuration file and hands over the results
to all hooked config_action_handler objects.

*class controller*: the controller does a lot of the work within newsbeuter. It
is the connector between view and cache. It parses the command line option,
controls the configuration parser and hands over data to the view. It also
contains code to control the reloading of feeds.

*class downloadthread*: a thread that does nothing but start a reload operation
for all feeds. Derived from thread.

*class htmlrenderer*: takes a string and renders the HTML it contains into a
textual representation. It also extracts URLs from a and img tag for later use.

*class keymap*: hooks into the configuration parser and manages all the
keymapping. Additionally, it generates the keymapping information with hint
texts for the view.

*class logger*: helper class that manages the optional logging to a text file.
Logging can be enabled by developers (see below).

*class mutex*: a C++ wrapper around the pthread mutex.

*class reloadthread*: similar to downloadthread, but starts a reload every n
minutes (configurable).

*class rss_feed*: represents an RSS feed, including RSS url, page link, title,
author, description and RSS items (articles). Uses the cache to persist itself.
Internally, all text data (especially the title and the author) are stored as
UTF-8 strings, but the getters return data that matches the current locale
charset setting.

*class rss_item*: represents an RSS item (article), including link to the
article, title, author, publication date and description. Internally, all text
data (especially the title, the author and the description) are stored as UTF-8
strings, but the getters return data that matches the current locale charset
setting.

*class stflpp*: a C++ wrapper around STFL. STFL is the ncurses widget library
that newsbeuter heavily relies upon.

*class thread*: a wrapper around Unix pthreads.

*class urlreader*: manages reading and writing the urls file, including handling
of tags.

*class utils*: contains several static utility functions, such as a tokenizer,
the lock file code and a text converter that builds upon iconv.

*class view*: the class that draws the user interface. It manages a stack of
so-called "form actions", each of which represents one dialog. The view class
delegates all received user input events to the correct form action.

*class formaction*: the abstract base class for all form actions. Currently, the
following formaction-derived classes exist:
- feedlist_formaction
- itemlist_formaction
- itemview_formaction
- urlview_formaction
- filebrowser_formaction
- help_formaction
- selecttag_formaction

*class tagsouppullparser*: Parses virtually everything that vaguely looks like
XML/HTML tags, even when totally invalid, and provides the parsing result as a
continuous stream of tokens (tags and text). It is solely used by the
htmlrenderer class.

=== Interaction

**Status of this section:** yet unwritten :)

TODO: describe interaction between classes.


== Design Decisions

=== Use text file as configuration

**Status of this section:** the idea holds (January 2021).

The "classical" text tools, like vim, slrn and mutt, are all configurable
solely via text files. Newsbeuter follows the same spirit, especially since the
other prominent RSS feed readers for the text console primarily encourage
configuration via an often crude user interface within the application itself.
The consequence for newsbeuter is: no configuration via the user interface, but
solely via configuration files. Text editors are easier to handle than some
crude menus that are somehow hard to use.

=== Keep a good balance of customizability

**Status of this section:** the idea holds, but Newsboat arguably failed this
goal (January 2021).

The problem with user wishes is that too many people demand a possibility to
customize this bell or that whistle within newsbeuter. Often, these
possibilities only have a very limited purpose, and their value is in no
relation to the added complexity of the code. Every customization needs to be
tested, and means a lot more testing whenever some related code changes.

The code shouldn't get too bloated, it should be kept straight-forward and easy
to read. With too much customizability, this goal would be in danger.

=== Why C++ and not C, [insert your favorite language], ...?

**Status of this section:** partially holds, but we're moving to Rust (January
2021).

C++ has many advantages compared to other programming languages.

- C++ is backwards-compatible to C. That means we can theoretically use all the
  C libraries.
- C++ makes it easier to structure your program in an object-oriented way, and 
  helps maintain inheritance hierarchies without a lot of fuzz.
- C++ compiles to fast, native code.
- C++ comes with an extensive standard library (see next section).
- C++ is widespread.
- C++ on Linux/Unix systems does not require any exotic compilers to be 
  installed in order to compile newsbeuter. g++ (part of GCC) is enough.

These were the reasons why C++ was initially chosen, and it proved to be a 
useable language during the development process.


=== Use the full potential of modern C++

**Status of this section:** mostly holds, but new stuff should be written in
Rust if possible (January 2021).

The C++ standard library comes with an extensive set of algorithms and data
structures. Developers are encouraged to use especially the data structures,
because the available container classes are standardized, their behaviour and
usage is well-documented, and makes it possible to keep the overall logic
at a pretty high level. More complex things that can only be done in C (like
special system calls) /should/ be encapsulated by a wrapper class in order to
avoid potential mis-use of low-level functions and data structures. Good
examples for wrapping low-level stuff are *class rss_feed*, *class rss_item*
and *class stflpp*.

== Tips and Tricks

=== Getting a detailed debug log

**Status of this section:** up-to-date (January 2021).

If you want to get a detailed debug log from newsbeuter, you only need to run
newsbeuter with special parameters:

  newsbeuter -d log.txt -l 6

Some of this output doesn't make sense very much unless you know the source
code, so it's only helpful for developers.

=== Use (and extend) the unit tests

**Status of this section:** up-to-date (January 2021).

C++ tests are in the _test_ subdirectory. They're using
https://github.com/catchorg/Catch2/tree/devel/docs[the Catch2 framework]. Rust
tests are split: unit tests are in `mod tests` sub-modules under
_rust/libnewsboat/src_, while integration tests are under
_rust/libnewsboat/tests_. Rust's test runner is multi-threaded, so we use
integration testing when we want to do the checks in a separate process.

In addition to `TMPDIR=/dev/shm make -j5 PROFILE=1 check` incantation that's
already explained in the contributing guidelines, you should know about the
`NEWSBOAT_RUN_IGNORED_TESTS` variable. Like `PROFILE`, it can be set to `1`.
When set, it enables tests that require some additional prerequisites:

- locales:
    - `en_US.UTF-8`
    - `ru_RU.CP1251`
    - `ru_RU.KOI8-R`

== Keys

=== Unused keys

**Status of this section:** probably outdated (January 2021).

	      _b_____hi_____________wxyz
	      _B_____HI__LM_____STUVWXYZ
	Ctrl- A__D___HIJ____O_Q_S___W_Y_

Used special keys: ENTER ? / # + - : , $ |
