...

Package mesh

import "github.com/Diarkis/diarkis/mesh"
Overview
Index
Subdirectories

Overview ▾

Package mesh ▷

Mesh - Server to server internal communications

Mesh manages and controls all internal server to server communications.

Configurations

Mesh configurations are explained below.

{
  "address": "127.0.0.1",
  "nic": "eth0",
  "port": "8000",
  "marsAddress": "127.0.0.1",
  "marsPort": "6789",
  "retryInterval": 1000,
  "retryTimeout": 3000,
  "strictCheckForExternal": false
  "sendInterval": 0
}

▶︎ Configuration Values

address - Default is "127.0.0.1"

Address for mesh network server to bind with.

nic - Default is "eth0"

Name of a network interface to retrieve address from.

port - Default is "8000"

Port for mesh network server to bind with. Starts from the given port and scans to find an open poet to bind.

marsAddress - Default is "127.0.0.1"

MARS server address. Can be overwritten by env DIARKIS_MARS_ADDR.

marsPort - Default is "6789"

MARS server port. Can be overwritten by env DIARKIS_MARS_PORT.

NOTE - You may use either address or nic in your configurations. If none is used, the default is localhost.

retryInterval - Default is 1000ms

RUDP retry interval in milliseconds

retryTimeout - Default is 3000ms

RUDP retry timeout.

strictCheckForExternal - Default is false

When this configuration is set to true, all in coming mesh messages will be validated to make sure the sender of each message comes from a node that is part of the cluster.

sendInterval - Message buffering interval in milliseconds. Default is 0ms

If sendInterval is set to be greater than 17 (17 is the minimum value), it will buffer the mesh messages.

Request and response sylte server to server communication

Mesh module allows you to send a request to a server in the Diarkis cluster and have the server send a response back.

▶︎ Preparation

You must have a receiver function defined in order to handle requests from another server.

// This must be called in the place where it gets called once during the server start process.
// It does not have to be a closure function here.
diarkis.OnStart(func() {
  // This is how to register the request handler for a specific request command ID
  mesh.HandleCommand(myAwesomeRequestCommandID, func(req map[string]interface{}) ([]byte, error) {

    // req map contains the request parameters send from another server.

    // The first returned value which is a []byte will be the response data sent to the sender server.

    // By returning an error, the request sender server will receive the response as an error.

    // response data caan be a struct as well
    myAwsomeResponseDataMap := make(mapp[string]interface{})

    myAwsomeResponseDataMap["response"] = "I handled your request beautifully!"

    bytes, err :=  mesh.CreateResponseBytes(myAwsomeResponseDataMap)

    return bytes, err

  })
})

▶︎ Send a request and receive the response

type MyAwsomeRequestData struct {
  Greeting string `json:"greeting"`
}

requestData := &MyAwesomeRequestData{ Greeting: "Hello" }

// The callback receives the request response.
mesh.SendRequest(myAwesomeRequestCommandID, anotherServerMeshAddress, requestData, func(err error, resp map[string]interface{}) {

  if err != nil {
    // The request caused an error...
  }

})

Utility functions that are useful

▶︎ How to find out the server type of the current server that you are operating on

myServerNodeType = mesh.GetNodeType()

▶︎ How to find out the mesh address (internal address for server to server communication) of the current server that you are operating on

myMeshAddr := mesh.GetMyEndPoint()

▶︎ How to find out the public address (internet facing address) of the current server that you are operating on

myPublicAddr := mesh.GetMyNodeEndPoint()

▶︎ Get all mesh addresses of a specific server type

addrs := mesh.GetNodeAddressesByType("$(server_node_type)")

Index ▾

func Command(commandID uint16, callback func(map[string]interface{}) (map[string]interface{}, error))
func CreateRequestData() *util.Parcel
func CreateReturnBytes(data interface{}) ([]byte, error)
func CreateSendBytes(data interface{}) ([]byte, error)
func GetBool(data map[string]interface{}, name string) bool
func GetBytes(data map[string]interface{}, name string) []byte
func GetFloat64(data map[string]interface{}, name string) float64
func GetInt(data map[string]interface{}, name string) int
func GetInt16(data map[string]interface{}, name string) int16
func GetInt32(data map[string]interface{}, name string) int32
func GetInt64(data map[string]interface{}, name string) int64
func GetInt8(data map[string]interface{}, name string) int8
func GetMyEndPoint() string
func GetMyNodeEndPoint() string
func GetMyNodeType() string
func GetMyNodeValue(name string) interface{}
func GetNodeAddresses(ignoreList []string) []string
func GetNodeAddressesByRole(nRole string) []string
func GetNodeAddressesByType(nType string) []string
func GetNodeEndPoint(nodeAddr string) string
func GetNodeRoleByType(nType string) string
func GetNodeType() string
func GetNodeTypeByAddress(addr string) string
func GetNodeTypes() []string
func GetNodeValue(nodeAddr string, name string) interface{}
func GetNodeValues(nodeAddr string) map[string]interface{}
func GetSharedData(key string) (int16, bool)
func GetString(data map[string]interface{}, name string) string
func GetUint16(data map[string]interface{}, name string) uint16
func GetUint32(data map[string]interface{}, name string) uint32
func GetUint64(data map[string]interface{}, name string) uint64
func GetUint8(data map[string]interface{}, name string) uint8
func Guest(confpath string)
func HandleCommand(cmd uint16, cb func(requestData map[string]interface{}) ([]byte, error))
func IsCommandDuplicate(commandID uint16) bool
func IsMyNodeOffline() bool
func IsMyNodeOnline() bool
func IsMyNodeTaken() bool
func IsNodeOffline(nodeAddr string) bool
func IsNodeOnline(nodeAddr string) bool
func IsNodeTaken(nodeAddr string) bool
func OnInitialSync(cb func())
func OnSharedDataRemove(cb func(string))
func OnSharedDataUpdate(cb func(string, int16))
func OnUpdate(handler func())
func OnUpdated(handler func())
func RemoveSharedData(key string) bool
func Send(commandID uint16, addr string, data interface{}) error
func SendMany(commandID uint16, addrs []string, data interface{}, limit int) error
func SendRequest(commandID uint16, addr string, data interface{}, callback func(err error, responseData map[string]interface{}))
func SetAppName(name string)
func SetEndPoint(endPoint string)
func SetNodeRole(role string)
func SetNodeType(_nodeType string)
func SetNodeValue(name string, val interface{})
func SetSharedData(key string, value int16) bool
func SetStatus(status int)
func Setup(confpath string)
func StartAsMARS(confpath string)
func USend(commandID uint16, addr string, data interface{}) error
func USendMany(command uint16, addrs []string, data interface{}, limit int) error
func ValidateDataType(data interface{}) bool
type Node
    func GetNode(nodeAddr string) *Node
    func GetNodeByEndPoint(endPoint string) *Node
type SendData

func Command

func Command(commandID uint16, callback func(map[string]interface{}) (map[string]interface{}, error))

Command [DEPRECATED] defines a handling function for a mesh network messsage as a command Command is replaced by HandleCommand - Command messages are sent by SendRequest, SendMany, USend, USendMany

func CreateRequestData

func CreateRequestData() *util.Parcel

CreateRequestData returns an instance of util.Parcel to be used for SendRequest, SendMany, USendMany, Send, and USend.

func CreateReturnBytes

func CreateReturnBytes(data interface{}) ([]byte, error)

CreateReturnBytes converts a map into a byte array for HandleCommand handler's return value

func CreateSendBytes

func CreateSendBytes(data interface{}) ([]byte, error)

CreateSendBytes converts a map into a byte array for Send, USend, SendMany, USendMany, and SendRequest

func GetBool

func GetBool(data map[string]interface{}, name string) bool

GetBool returns a bool from the mesh network request/response data

func GetBytes

func GetBytes(data map[string]interface{}, name string) []byte

GetBytes returns a byte array from the mesh network request/response data

func GetFloat64

func GetFloat64(data map[string]interface{}, name string) float64

GetFloat64 returns a float64 from the mesh network request/response data

func GetInt

func GetInt(data map[string]interface{}, name string) int

GetInt returns an int from the mesh network request/response data

func GetInt16

func GetInt16(data map[string]interface{}, name string) int16

GetInt16 returns an int16 from the mesh network request/response data

func GetInt32

func GetInt32(data map[string]interface{}, name string) int32

GetInt32 returns an int32 from the mesh network request/response data

func GetInt64

func GetInt64(data map[string]interface{}, name string) int64

GetInt64 returns an int64 from the mesh network request/response data

func GetInt8

func GetInt8(data map[string]interface{}, name string) int8

GetInt8 returns an int8 from the mesh network request/response data

func GetMyEndPoint

func GetMyEndPoint() string

GetMyEndPoint Returns its own mesh network endpoint that is used for internal server-to-server communication.

[IMPORTANT] Send, USend, SendMany, USendMany, and SendRequest needs this mesh network endpoint.

The returned address is external address for clients.

[NOTE] Uses mutex lock internally.

func GetMyNodeEndPoint

func GetMyNodeEndPoint() string

GetMyNodeEndPoint returns the server endpoint (internet) its own node

The returned address is external address for clients.

[NOTE] Uses mutex lock internally.

func GetMyNodeType

func GetMyNodeType() string

GetMyNodeType Returns the node type of itself

func GetMyNodeValue

func GetMyNodeValue(name string) interface{}

GetMyNodeValue returns a metadata value of the node

[NOTE] Uses mutex lock internally.

func GetNodeAddresses

func GetNodeAddresses(ignoreList []string) []string

GetNodeAddresses Returns all internal server-to-server communication node addresses.

[NOTE] Uses mutex lock internally.

func GetNodeAddressesByRole

func GetNodeAddressesByRole(nRole string) []string

GetNodeAddressesByRole Return the mesh network endpoints for internal server-to-server communication of nodes by server role (role is based on network protocol)

[IMPORTANT] Send, USend, SendMany, USendMany, and SendRequest needs this mesh network endpoint.

[NOTE] Uses mutex lock internally.

func GetNodeAddressesByType

func GetNodeAddressesByType(nType string) []string

GetNodeAddressesByType Return the mesh network endpoints for internal server-to-server communication of nodes by type.

[IMPORTANT] Send, USend, SendMany, USendMany, and SendRequest needs this mesh network endpoint.

Server type can be changed by the application.

The returned list of addresses contain offline and taken nodes as well.

[NOTE] Uses mutex lock internally.

func GetNodeEndPoint

func GetNodeEndPoint(nodeAddr string) string

GetNodeEndPoint returns the public server endpoint (internet) of a node by its mesh node address - only TCP, UDP, and HTTP

[IMPORATNT] Endpoint that is used to communicate with the clients over internet.

The returned address is external address for clients.

[NOTE] Uses mutex lock internally.

func GetNodeRoleByType

func GetNodeRoleByType(nType string) string

GetNodeRoleByType Returns node role by node type.

[NOTE] Uses mutex lock internally.

Possible values for nType:

func GetNodeType

func GetNodeType() string

GetNodeType returns the server type of the server

func GetNodeTypeByAddress

func GetNodeTypeByAddress(addr string) string

GetNodeTypeByAddress returns the server type of the given node by mesh address (internal address)

[IMPORTANT] Send, USend, SendMany, USendMany, and SendRequest needs this mesh network endpoint.

[NOTE] Uses mutex lock internally.

func GetNodeTypes

func GetNodeTypes() []string

GetNodeTypes returns all node types currently in the Diarkis cluster

[NOTE] Uses mutex lock internally.

func GetNodeValue

func GetNodeValue(nodeAddr string, name string) interface{}

GetNodeValue Returns a metadata value of a node by its address

[NOTE] Uses mutex lock internally.

Parameter

nodeAddr - External address (address for clients) of the node.
name     - Name of the value.

func GetNodeValues

func GetNodeValues(nodeAddr string) map[string]interface{}

GetNodeValues Returns all metadata values of a node by its address

[NOTE] Uses mutex lock internally.

Parameter

nodeAddr - External address (address for clients) of the node.

func GetSharedData

func GetSharedData(key string) (int16, bool)

GetSharedData returns synchronized shared data by its key.

func GetString

func GetString(data map[string]interface{}, name string) string

GetString returns a string from the mesh network request/response data

func GetUint16

func GetUint16(data map[string]interface{}, name string) uint16

GetUint16 returns a uint16 from the mesh network request/response data

func GetUint32

func GetUint32(data map[string]interface{}, name string) uint32

GetUint32 returns a uint32 from the mesh network request/response data

func GetUint64

func GetUint64(data map[string]interface{}, name string) uint64

GetUint64 returns a uint64 from the mesh network request/response data

func GetUint8

func GetUint8(data map[string]interface{}, name string) uint8

GetUint8 returns a uint8 from the mesh network request/response data

func Guest

func Guest(confpath string)

Guest starts mesh network as a guest meaning that the process can only send and receive messages but does not become part of the mesh node

func HandleCommand

func HandleCommand(cmd uint16, cb func(requestData map[string]interface{}) ([]byte, error))

HandleCommand defines a handling function for a mesh network messsage as a command

Parameters

cmd - Internal message command ID.
cb  - Callback associated to the command ID.

func IsCommandDuplicate

func IsCommandDuplicate(commandID uint16) bool

IsCommandDuplicate returns true if commandID is used elsewhere

func IsMyNodeOffline

func IsMyNodeOffline() bool

IsMyNodeOffline returns true if the node is offline (received SIGTERM)

[NOTE] Uses mutex lock internally.

When the node is offline, the connected clients will be raising OnOffline event

func IsMyNodeOnline

func IsMyNodeOnline() bool

IsMyNodeOnline returns false if the node is offline (received SIGTERM)

[NOTE] Uses mutex lock internally.

When the node is offline, the connected clients will be raising OnOffline event

func IsMyNodeTaken

func IsMyNodeTaken() bool

IsMyNodeTaken returns true if the node is marked as taken

[NOTE] Uses mutex lock internally.

When the node is taken, the server will not accept new client connectios, but allows the connected clients to stay connected.

func IsNodeOffline

func IsNodeOffline(nodeAddr string) bool

IsNodeOffline Returns true if the given node is offline

[NOTE] Uses mutex lock internally.

When the node is offline, the connected clients will be raising OnOffline event

func IsNodeOnline

func IsNodeOnline(nodeAddr string) bool

IsNodeOnline Returns true if the given node is online

[NOTE] Uses mutex lock internally.

When the node is offline, the connected clients will be raising OnOffline event

func IsNodeTaken

func IsNodeTaken(nodeAddr string) bool

IsNodeTaken returns false if the node is taken (mesh.SetStatus(util.MeshStatusTaken))

[NOTE] Uses mutex lock internally.

When a node is thaken, the taken node will not be returned by HTTP server as available server.

func OnInitialSync

func OnInitialSync(cb func())

OnInitialSync assigns a callback on the first sync with MARS

func OnSharedDataRemove

func OnSharedDataRemove(cb func(string))

OnSharedDataRemove assigns a callback on shared data removal

func OnSharedDataUpdate

func OnSharedDataUpdate(cb func(string, int16))

OnSharedDataUpdate assigns a callback on shared data update

func OnUpdate

func OnUpdate(handler func())

OnUpdate Registers a callback on announcer update event

[NOTE] Uses mutex lock internally.

func OnUpdated

func OnUpdated(handler func())

OnUpdated Registers a callback on announcer updated event

[NOTE] Uses mutex lock internally.

func RemoveSharedData

func RemoveSharedData(key string) bool

RemoveSharedData removes the given shared key and propates the removeal tto all server nodes in the cluster.

The propagation of the shared data may take some time.

[IMPORTANT] Updated value may suffer from race conditing

if multiple server nodes attempt to update the same key.

func Send

func Send(commandID uint16, addr string, data interface{}) error

Send Sends a mesh network message to a given node - addr = <address>:<port>

func SendMany

func SendMany(commandID uint16, addrs []string, data interface{}, limit int) error

SendMany Sends a mesh network message to multiple nodes

commandID - Mesh message command ID addrs - A list of mesh node addreses to send the message to data - Data to be sent limit - Maximum number of node to send the message at a time

SendMany propagates data to multiple servers.

The diagram below show how SendMany works with limit=2 and send data to 6 servers:

With the example below, SendMany can reach all servers with 2 jumps.

                                                           ┌──────────┐
                                                     ┌────▶︎│ Server D │
                                                     │     └──────────┘
                               ┌──────────┐ <2> Send │
                           ┌──▶︎│ Server B │──────────┤     ┌──────────┐
                           │   └──────────┘          └────▶︎│ Server E │
┌──────────┐ <1> Send      │                               └──────────┘
│ Server A │ ──────────────┤                               ┌──────────┐
└──────────┘               │   ┌──────────┐          ┌────▶︎│ Server F │
                           └──▶︎│ Server C │──────────┤     └──────────┘
                               └──────────┘          │
                                                     │     ┌──────────┐
                                                     └────▶︎│ Server G │
                                                           └──────────┘

func SendRequest

func SendRequest(commandID uint16, addr string, data interface{}, callback func(err error, responseData map[string]interface{}))

SendRequest Sends a mesh network requesst to another node - addr = <address>:<port> and expects a response back from it

Parameters

commandID - Pre-defined command ID that coresponds with user defined custom handler function(s).
            You may define multiple handler functions for a command ID.
addr      - Target node address (inernal) to send the request to.
data      - Request data to be sent to the handler function(s).
            The valid data type of data must be the following two types only:
            1. map[string]interface{}
            2. struct
callback  - The callback function to be invoked when the response comes back.

SendRequest allows you to execute a pre-defined function on another server and expects a response back from the server.

Error Cases

┌───────────────────┬─────────────────────────────────────────────────────────────────────────┐
│ Error             │ Reason                                                                  │
╞═══════════════════╪═════════════════════════════════════════════════════════════════════════╡
│ Invalid data type │ Input data type must be either a struct or map[string]interface{}.      │
│ Handler error     │ Handler function of the request returned an error.                      │
│ Network error     │ Mesh network error. Failed to send or receive server-to-server message. │
╘═══════════════════╧═════════════════════════════════════════════════════════════════════════╛

The diagram below show how SendRequest works:

┌──────────┐ <1> Send request  ┌──────────┐
│ Server A │ ─────────────────▶︎│ Server B │
│          │ ◀︎──────────────── │          │
└──────────┘ <2> Send Response └──────────┘

func SetAppName

func SetAppName(name string)

SetAppName creates a name space within the mars-server in order to have multiple diarkis clusters

func SetEndPoint

func SetEndPoint(endPoint string)

SetEndPoint Sets its own end point

[NOTE] Uses mutex lock internally.

func SetNodeRole

func SetNodeRole(role string)

SetNodeRole [INTERNAL USE ONLY]

func SetNodeType

func SetNodeType(_nodeType string)

SetNodeType replaces the default server type with the given custom server type.

Default Server Types - HTTP, UDP, TCP, WS

func SetNodeValue

func SetNodeValue(name string, val interface{})

SetNodeValue Sets a metadata value to a node

[NOTE] Uses mutex lock internally.

func SetSharedData

func SetSharedData(key string, value int16) bool

SetSharedData updates a shared data to be propagated to all server nodes in the cluster.

The propagation of the shared data may take some time.

[IMPORTANT] Updated value may suffer from race conditing

if multiple server nodes attempt to update the same key.

func SetStatus

func SetStatus(status int)

SetStatus Sets its status

func Setup

func Setup(confpath string)

Setup Loads a configuration file into memory - pass an empty string to load nothing

func StartAsMARS

func StartAsMARS(confpath string)

StartAsMARS Starts the server as MARS server - For now this is meant to be used for mars-server ONLY

func USend

func USend(commandID uint16, addr string, data interface{}) error

USend Sends an unreliable message to a given node - addr = <address>:<port>

func USendMany

func USendMany(command uint16, addrs []string, data interface{}, limit int) error

USendMany Sends an unreliable mesh network message to multiple nodes

commandID - Mesh message command ID addrs - A list of mesh node addreses to send the message to data - Data map to be sent limit - Maximum number of node to send the message at a time

USendMany propagates data to multiple servers.

The diagram below show how USendMany works with limit=2 and send data to 6 servers:

With the example below, USendMany can reach all servers with 2 jumps.

                                                           ┌──────────┐
                                                     ┌────▶︎│ Server D │
                                                     │     └──────────┘
                               ┌──────────┐ <2> Send │
                           ┌──▶︎│ Server B │──────────┤     ┌──────────┐
                           │   └──────────┘          └────▶︎│ Server E │
┌──────────┐ <1> Send      │                               └──────────┘
│ Server A │ ──────────────┤                               ┌──────────┐
└──────────┘               │   ┌──────────┐          ┌────▶︎│ Server F │
                           └──▶︎│ Server C │──────────┤     └──────────┘
                               └──────────┘          │
                                                     │     ┌──────────┐
                                                     └────▶︎│ Server G │
                                                           └──────────┘

func ValidateDataType

func ValidateDataType(data interface{}) bool

ValidateDataType returns true if the given data type is either a pointer to a struct or map[string]interface {}

type Node

Node a mesh network node data structure

type Node struct {
    Addr   string
    Type   string
    Role   string
    Values map[string]interface{}
    Self   bool
}

func GetNode

func GetNode(nodeAddr string) *Node

GetNode Returns a node by its address

[NOTE] Uses mutex lock internally.

Parameters

nodeAddr - External address (address for clients) of the node.

The returned *Node contains internal node address as *Node.Addr.

func GetNodeByEndPoint

func GetNodeByEndPoint(endPoint string) *Node

GetNodeByEndPoint Returns a node by its server end point - only WebSocket, TCP, UDP, and HTTP

[NOTE] Uses mutex lock internally.

type SendData

SendData represents internally used data for mesh communication

type SendData struct {
    Payload interface{} `json:"payload"`
    Cmd     uint16      `json:"cmd"`
    Limit   int         `json:"limit"`
    Branch  []string    `json:"branch"`
    UBranch []string    `json:"ubranch"`
}

Subdirectories