neuropil API

Note

The neuropil API makes use of types defined in the standard C headers stdint.h, stdbool.h and stddef.h.

neuropil.h

typedef void np_context

An opaque object that denotes a neuropil application context.

np_context *np_new_context(struct np_settings *settings)

Creates a new neuropil application context.

Parameters:

  • settings – a np_settings structure used to configure the

application context. :return: a pointer to the newly created application context.

typedef void np_subject

An blake2b obfuscated message subject to transport data

void np_default_settings(struct np_settings *settings)

Initializes a np_settings structure to the default settings.

Parameters:

  • settings – a pointer to the np_settings structure to

be initialized.

struct np_settings

The np_settings structure holds various run-time preferences available to neuropil.

uint32_t n_threads

Controls the maximum number of threads used by neuropil at any given time. The default is 3.

char log_file[256]

Pathname of a file that neuropil will log to. The default is a "<timestamp>_neuropil.log" where timestamp is a decimal millisecond UNIX timestamp.

uint8_t leafset_size

specifies the size of the leafset table (DHT entries near to our own hash value)

uint16_t jobqueue_size

The size of the internally used jobqueue. The default is 512 entries, depending on the number of threads this should be sufficient for many use cases. High throuput cloud nodes could need larger jobqueues.

Identity management

struct np_token *np_new_identity(np_context* ac, double
expires_at, uint8_t secret_key[NP_SECRET_KEY_BYTES])

Creates a new neuropil identity.

Parameters:

  • ac – a neuropil application context.

  • expires_at – expiry date of the identity in seconds since the Unix

epoch. :param secret_key: the secret key used by the identity. If NULL is supplied a random key is generated. :return: an identity token.

enum np_return np_set_identity(np_context* ac, struct np_token
identity)

Sets the identity used by the neuropil node.

Parameters:

  • ac – a neuropil application context.

  • identity – the identity to use.

Returns:

np_ok on success.

struct np_token *np_use_identity(np_context* ac, struct np_token
identity)

imports an identity into the running process and uses it as it’s own identity

Parameters:

  • ac – a neuropil application context.

  • identity – a valid token structure which should be used as the

identity. The secret key must be present, otherwise the operation has no effect

struct np_token *np_use_token(np_context* ac, struct np_token
token)

imports an token into the running process and uses it. The imported token can be a node, an different identity, a message intent token or an accounting token. The library with set the status of the token to AUTHENTICATED and AUTHORIZED, as the user requested to import the token

Parameters:

  • ac – a neuropil application context.

  • identity – a valid token structure which should be used as the

identity. The secret key must not be present. If the token is not valid (i.e. expired, signature broken), then it will be rejected

Starting up

enum np_return np_listen(np_context* ac, const char* protocol,
const char* host, uint16_t port, const char * dns_name)

Binds a neuropil application context to a listening address.

Parameters:

  • ac – a neuropil application context.

  • protocol – a string denoting the underlying protocol to be used.

Currently, only “udp4” is supported. :param host: the hostname to listen on. I.e., “localhost” to listen on the loopback interface. :param port: the port to listen on. If port is zero, the default port 3141 is used. :param dns_name: the dns name to publish, same as hostname if NULL. :return: np_ok on success.

Status

Meaning

np_invalid_argument

Host could not be resolved.

np_invalid_operation

No identity is set for the application

context.
enum np_return np_get_address(np_context* ac, char* address,
uint32_t max)

Gets the absolute address of the neuropil node within the overlay network.

Parameters:

  • ac – a neuropil application context.

  • address – a pointer to the address string to be written to.

  • max – the size in bytes of address. Should be large enough to hold

the resulting address string. The required space depends on the node’s host name (i.e., 1000 bytes should be more than enough for most uses.) :return: np_ok on success.

enum np_return np_join(np_context *ac, const char *address)

Adds a bootstrap node to be used by this node to join the neuropil network.

Parameters:

  • ac – a neuropil application context.

  • address – a string that denotes an absolute address as obtained by

np_get_address(). :return: np_ok on success.

Status

Meaning

np_invalid_argument

Address is malformed.

Sending and receiving messages

enum np_return np_generate_subject(np_subject subject, const
char* text, size_t length)

Creates the binary representation of a message subject. This is an re-entrant version, using the same subject field will use the existing np_subject as a seed for the text.

Parameters:

  • subject – the final np_subject field that can be used in later calls

to the library :param text: the text that the subject should be based on :param length: the length of text in bytes. :return: np_ok on success.

enum np_return np_send(np_context* ac, np_subject subject, const
uint8_t* message, size_t length)

Sends a message on a given subject.

Parameters:

  • ac – a neuropil application context.

  • subject – the subject to send on.

  • message – a pointer to a buffer containing the message to be sent. The

message could be, for instance, encoded using MessagePack. :param length: the length of message in bytes. :return: np_ok on success.

enum np_return np_add_receive_cb(np_context* ac, np_subject
subject, np_receive_callback callback)

Adds a callback to be executed when receiving a message on a given subject. It is possible to add more than one receive callback for a given subject, in which case they are run in the order in which they were added.

Parameters:

  • ac – a neuropil application context.

  • subject – the subject to receive on.

  • callback – a pointer to a function of type

np_receive_callback that denotes the callback to be added. :return: np_ok on success.

bool np_receive_callback(struct np_message *message)

Receive callback function type to be implemented by neuropil applications. A message receipt is considered to be acknowledged if all receive callbacks associated with the subject returned (true). Once a receive callback returns (false), the message is considered rejected and no further callbacks for the subject are executed.

Parameters:

  • message – a pointer to a np_message structure.

Returns:

a boolean that indicates if the receipt was acknowledged

(true) or rejected (false.)

struct np_message

Structure that holds a received message and some metadata about that message.

uint8_t *data

A pointer to a buffer that contains the received message.

size_t data_length

The length of data in bytes.

char uuid[NP_UUID_BYTES]

A universally unique identifier for the message.

np_id from

The identity fingerprint of the message.

np_id subject

The fingerprint of the message subject.

double received_at

Unix timestamp that denotes the time the message was received.

enum np_return np_set_mx_properties(np_context* ac, const char*
subject, struct np_mx_properties properties)

Configure message exchange semantics for a given subject. The default is best-effort message delivery without any attempt at retransmission and if delivered messages are guaranteed to be delivered once only.

Parameters:

  • ac – a neuropil application context.

  • subject – the subject to configure message exchange semantics on.

  • properties – a pointer to a np_mx_properties structure that

describes the semantics to be applied. :return: np_ok on success.

Status

Meaning

np_invalid_argument

The properties structure is invalid.
struct np_mx_properties

Structure that defines message the exchange semantics for a given subject.

np_mx_ackmode ackmode

Acknowledgement strategy used in message exchange. The default is NP_MX_ACK_NONE (i.e., fire and forget.)

np_mx_role role

role of the node for this data transfer. The default is not set. When calling np_send the library creates the role of NP_MX_PROVIDER, if calling np_add_receive_cb the role NP_MX_CONSUMER is used

np_mx_audience_type audience_type

the intended audience for this data transfer. see np_mx_audience_type for the specific meaning of each. The default is NP_MX_AUD_PUBLIC

np_mx_cache_policy cache_policy

Cache policy used for queuing inbound messages. The default is NP_MX_FIFO_REJECT (i.e., messages are delivered first in, first out, and messages that would overflow the queue will be rejected.)

uint8_t max_parallel

The maximum number of outbound messages that may be in-flight at a given moment. The default is one.

uint8_t max_retry

The maximum number of times a message will be resent after it has been rejected. The default is zero.

double intent_ttl
double intent_update_after

The duration of validity of issued message intents and the duration after which message intents are to be refreshed in seconds. Intent_update_after should always be less than intent_ttl. This setting impacts the fail-over latency between receivers leaving and joining the network. Senders will remain unaware of new receivers for up to intent_update_after seconds.

The default for intent_ttl is 30 seconds. The default for intent_update_after is 20 seconds

double message_ttl

Maximum duration for individual message delivery in seconds. The default is 20 seconds.

bool once_only

Boolean that indicates if messages should be ensured to be delivered once only. If this is false duplicate messages may be delivered to the application. The default is true.

enum np_mx_cache_policy
enum np_mx_ackmode
enum np_mx_role

Mode

Description

NP_MX_PROVIDER

node is the sender of messages

NP_MX_CONSUMER

node is the receiver of messages

NP_MX_PROSUMER

node will send an receiver messages on this

subject
enum np_mx_audience_type

Authentication and authorization

enum np_return np_set_authorize_cb(np_context* ac,
np_aaa_callback callback)

Sets the authorization callback used to control access to message exchanges. The provided callback is responsible for judging whether the identity that issued a given token is permitted to exchange messages over a given subject. If no authorization callback is set all message exchanges will be rejected.

Parameters:

  • ac – a neuropil application context.

  • callback – a pointer to a function of type np_aaa_callback

that denotes the callback to be set. :return: np_ok on success.

enum np_return np_set_authenticate_cb(np_context* ac,
np_aaa_callback callback)

Sets an additional authentication callback used to authenticate nodes. Such a callback can be used to extend the authentication provided by neuropil to further validate token based on application extensions. If no such callback is set only standard neuropil authentication is performed. Note that authenticated nodes are permitted to join the overlay network.

Parameters:

  • ac – a neuropil application context.

  • callback – a pointer to a function of type np_aaa_callback

that denotes the callback to be set. :return: np_ok on success.

bool np_aaa_callback(struct np_token *aaa_token)

AAA callback function type to be implemented by neuropil applications. These functions are to inspect and verify the contents of the aaa_token they are provided and either accept or reject the token.

Parameters:

  • aaa_token – a pointer to a np_token structure to be verified.

Returns:

a boolean that indicates if the token was accepted

(true) or rejected (false.)

Token

struct np_token

A record used for authentication, authorization and accounting purposes. When np_token records are transmitted or received over the network they are accompanied by a cryptographic signature that must match the token’s public_key field. A token received through the neuropil API are guaranteed to be authentic: i.e., their integrity is validated. Applications are responsible to verify the issuer of a given token as denoted by the public_key.

np_id realm

Optionally, token can specify a third party authority that governs the validity of token in a realm.

np_id issuer

The fingerprint of the identity that issued the token.

char subject[255]

A subject that denotes the token’s purpose.

np_id audience

A fingerprint that denotes the intended audience of the token.

double issued_at
double not_before
double expires_at

Timestamps encoded as Unix time in seconds that denote issue date and validity duration of the token. These validity periods are validated by neuropil.

uint8_t extensions[]
size_t extension_length

A buffer of extension data of extension_length bytes represented as a MessagePack encoded map.

uint8_t public_key[NP_PUBLIC_KEY_BYTES]
uint8_t secret_key[NP_SECRET_KEY_BYTES]

The key pair associated with the token. Foreign token have the secret_key unset (all zero).

Fingerprints

void np_get_id(np_id (*id), char *string, size_t length)

Computes the fingerprint (or overlay address) of a serialized object.

Parameters:

  • id – a np_id to be written.

  • string – the data to be hashed.

  • length – the length of the input data in bytes. If length is

zero, string is expected to be a zero-terminated string.

np_get_id() constructs fingerprints by means of a cryptographic, one-way hash function. Hence, fingerprints are unique, unforgeable object identities. Given an object, any party can compute its unique fingerprint, but no party is able to forge an object that hashes to a particular fingerprint.

typedef uint8_t np_id[NP_FINGERPRINT_BYTES]

The type np_id denotes both a fingerprint and a virtual address in the overlay network implemented by neuropil. It is represented as a consecutive array of NP_FINGERPRINT_BYTES bytes.

Running your application

enum np_return np_run(np_context *ac, double duration)

Runs the neuropil event loop for a given application context for a specified duration*. During the execution of the event loop incoming and outgoing messages are transmitted and received, and the associated callbacks are executed.

Parameters:

  • ac – a neuropil application context.

  • duration – the duration in seconds allotted to execute the event loop.

If duration is zero np_run() will return as soon as it has processed all outstanding events. :return: np_ok on success.

Detecting errors

enum np_return

This type denotes the set of status codes returned by various functions in the neuropil API. Possible values include:

In order to accurately interpret error codes refer to the documentation of the specific function in question.

Constants

size_t NP_SECRET_KEY_BYTES
size_t NP_PUBLIC_KEY_BYTES

Constants that denote the lengths in bytes of the private and public key parts used by neuropil, as found in np_token.

size_t NP_FINGERPRINT_BYTES

Constant that denotes length in bytes of both fingerprints and virtual addresses in the overlay network implemented by neuropil. Specifically, this is the size of np_id.

size_t NP_UUID_BYTES

Constant that denotes the length in bytes of message UUID_s.

neuropil_data.h

@brief Initialises the provided memory allocation as datablock

@param block The memory block provided by the user (malloc’ed ) @param block_length The memory block size @return enum np_data_return

@brief merges src datablock into dest datablock. overwrites existing keys

typedef void np_datablock_t

An opaque object that denotes a neuropil key/value store.

enum np_data_return np_init_datablock(np_datablock_t * block,
size_t block_length)

Creates a new neuropil key value store.

Parameters:

  • block – the memoryblock which should be initialized to contain

a np_datablock_t :param block_length: a size_t the size of the block input parameter :return: np_ok on success

struct np_data_conf

The configuration of a single key in an np_datablock_t

enum np_data_type

This type denotes the set of data types relevant for np_datablock_t key/value elements Possible values include:

=========================================== Status Meaning

=========================================== NP_DATA_TYPE_BIN Define the key to hold binary data in its value container NP_DATA_TYPE_INT Define the key to hold an integer in its value container NP_DATA_TYPE_STR Define the key to hold a string in its value container (Uses ASCII encoding) ================================================ ===========================================

bool np_data_iterate_cb(struct np_data_conf * out_data_config,
np_data_value * out_data, void * userdata)

Callback definition for the :c:function:`np_iterate_data` function

enum np_data_return np_iterate_data(np_datablock_t * block,
np_data_iterate_cb callback, void * userdata)

Iterates over every np_datablock_t key/value elements

neuropil_attributes.h

The purpose of this API is to enable the user to create and use arbitrary attributes on np_token and message intent level.

Attributes can then be e.g. used to:

  • publish searchable metadata (e.g. the location of a temperature sensor or

the cost of a data sample)

authentication. (see: NP_ATTR_IDENTITY_AND_USER_MSG)

  • many more …

enum np_msg_attr_type

This type denotes the set of scope levels relevant for a given attribute.

Possible values include:

NP_ATTR_USER_MSG The attribute will be available in every USER message. (end-to-end encrypted) NP_ATTR_IDENTITY The attribute will be appended to the identity np_token and will then be transfered to every node receiving a _NP.JOIN.REQUEST request. (end-to-end encrypted) NP_ATTR_INTENT The attribute will be appended to every _NP.MESSAGE.DISCOVER.RECEIVER and _NP.MESSAGE.DISCOVER.SENDER message. (transport encrypted) NP_ATTR_IDENTITY_AND_USER_MSG combine NP_ATTR_USER_MSG and NP_ATTR_IDENTITY NP_ATTR_INTENT_AND_USER_MSG combine NP_ATTR_USER_MSG and NP_ATTR_INTENT NP_ATTR_INTENT_AND_IDENTITY combine NP_ATTR_INTENT and NP_ATTR_IDENTITY ================================================ ======================================================================

enum np_return np_set_ident_attr_bin(struct np_token* ident,
enum np_msg_attr_type  inheritance, char key[255], unsigned char * bin, size_t
bin_length)

Sets an identity level wide attribute

Parameters:

this attribute :param key: the network wide identifier for this attribute :param bin: the attribute value as binary blob :param bin_length: the length of the blob :return: np_ok on success

enum np_return np_set_mxp_attr_bin(struct np_mx_properties*
prop, enum np_msg_attr_type  inheritance, char key[255], unsigned char * bin,
size_t bin_length)

Sets an message exchange level wide attribute

Parameters:

this attribute, only non NP_ATTR_IDENTITY… values are respected. :param key: the network wide identifier for this attribute :param bin: the attribute value as binary blob :param bin_length: the length of the blob :return: np_ok on success