initial commit

README.md

@@ -0,0 +1,43 @@
+# Node-RED Bitstamp Ticker Flow
+
+## Introduction
+This repository contains a Node-RED flow called `nodered_bitstamp_ticker_flow` authored by Lukas Goldschmidt. The flow fetches ticker information for all Bitstamp pairs and splits the information into tables for each pair. This data is then used in a trade bot application.
+
+## About Node-RED
+Node-RED is a powerful tool for building Internet of Things (IoT) applications with a focus on simplifying the 'wiring together' of code blocks to carry out tasks². It uses a visual programming approach that allows developers to connect predefined code blocks, known as 'nodes', together to perform a task². You can learn more about Node-RED [here](^1^).
+
+## About QuestDB
+QuestDB is a fast open-source time-series database with high-performance ingestion and SQL analytics⁶. It tackles difficulties when managing high throughput of time-based data and makes it easy to analyze the ingested data through simple SQL queries⁶. You can learn more about QuestDB [here](^7^).
+
+## Installation
+Node-RED can be installed on various platforms, including a local computer, a device such as a Raspberry Pi, or in the cloud¹. This flow is running on a Raspberry Pi 3 as an add-on of Home Assistant¹⁷. You can find detailed instructions on how to install Node-RED on a Raspberry Pi 3 [here](^9^) and [here](^10^).
+
+Once Node-RED is installed, you can import a flow into it. Detailed instructions on how to install a flow in Node-RED can be found [here](^13^) and [here](^15^).
+
+## Usage
+The only file in this repository is `bitstamp_ticker_flow.json`, which contains the Node-RED flow. After installing Node-RED and importing the flow, you can use it to fetch ticker information for all Bitstamp pairs.
+
+## Disclaimer
+This software comes with no warranty. The author is not responsible for any loss of money, time, sanity, or personal relationships as a result of using this code. Use at your own risk and amusement. Made under the influence of Cannabis and with the clever assistance of Bing AI and ChatGPT.
+
+Source: Conversation with Bing, 11/6/2023
+(1) Node-RED: Lecture 1 – A brief introduction to Node-RED. https://noderedguide.com/nr-lecture-1/.
+(2) Getting Started : Node-RED. https://nodered.org/docs/getting-started/.
+(3) Introduction to QuestDB | Baeldung. https://www.baeldung.com/java-questdb.
+(4) Introduction | QuestDB. https://questdb.io/docs/.
+(5) Getting Started : Node-RED. https://nodered.org/docs/getting-started/.
+(6) Getting Started - Home Assistant. https://www.home-assistant.io/getting-started/.
+(7) Running on Raspberry Pi : Node-RED. https://nodered.org/docs/getting-started/raspberrypi.
+(8) Install Node-RED on Raspberry Pi | Random Nerd Tutorials. https://randomnerdtutorials.com/install-node-red-raspberry-pi/.
+(9) Flows : Node-RED. https://nodered.org/docs/user-guide/editor/workspace/flows.
+(10) How to Export and Import Flows-Node-Red. https://stevesnoderedguide.com/copy-nodes-flows-node-red.
+(11) Node-RED - Wikipedia. https://en.wikipedia.org/wiki/Node-RED.
+(12) Node-RED. https://nodered.org/.
+(13) Steve’s Node-Red Guide. https://stevesnoderedguide.com/.
+(14) QuestDB - Database of Databases. https://dbdb.io/db/questdb.
+(15) How to install Node-RED and AutoStart on boot on Raspberry pi?. https://www.programmingboss.com/2023/03/how-to-install-node-red-and-autostart-on-boot-on-raspberryPi.html.
+(16) Install Node RED on Your Raspberry Pi : 4 Steps - Instructables. https://www.instructables.com/Install-Node-RED-on-Your-Raspberry-Pi/.
+(17) Creating your first flow : Node-RED. https://nodered.org/docs/tutorials/first-flow.
+(18) Library - Node-RED. https://flows.nodered.org/add/node.
+(19) Home Assistant. https://www.home-assistant.io/.
+(20) undefined. https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered%29.

bitstamp_ticker_flow.json

@@ -0,0 +1,318 @@
+[{
+        "id": "30e5a1df4ff2780a",
+        "type": "inject",
+        "z": "f6ebb8e404985534",
+        "name": "Once a minute",
+        "props": [{
+                "p": "payload"
+            }, {
+                "p": "topic",
+                "vt": "str"
+            }, {
+                "p": "timestamp",
+                "v": "",
+                "vt": "date"
+            }
+        ],
+        "repeat": "60",
+        "crontab": "",
+        "once": false,
+        "onceDelay": 0.1,
+        "topic": "Bitstamp Ticker",
+        "payload": "",
+        "payloadType": "date",
+        "x": 140,
+        "y": 80,
+        "wires": [["2bc9825b6248713e"]]
+    }, {
+        "id": "2bc9825b6248713e",
+        "type": "http request",
+        "z": "f6ebb8e404985534",
+        "name": "",
+        "method": "GET",
+        "ret": "obj",
+        "paytoqs": "ignore",
+        "url": "https://www.bitstamp.net/api/v2/ticker/",
+        "tls": "",
+        "persist": false,
+        "proxy": "",
+        "insecureHTTPParser": false,
+        "authType": "",
+        "senderr": false,
+        "headers": [],
+        "x": 390,
+        "y": 80,
+        "wires": [["dffe92840fbff3bb"]]
+    }, {
+        "id": "d8824452cc11c4e7",
+        "type": "debug",
+        "z": "f6ebb8e404985534",
+        "name": "DB Result",
+        "active": true,
+        "tosidebar": false,
+        "console": false,
+        "tostatus": true,
+        "complete": "payload",
+        "targetType": "msg",
+        "statusVal": "payload",
+        "statusType": "auto",
+        "x": 880,
+        "y": 260,
+        "wires": []
+    }, {
+        "id": "dffe92840fbff3bb",
+        "type": "function",
+        "z": "f6ebb8e404985534",
+        "name": "splitTickers",
+        "func": "var outputMsgs = [];\n\nmsg.payload.forEach(myFunction);\n\nfunction myFunction(value) {\n    outputMsgs.push({ payload: value, timestamp: msg.timestamp });\n}\n\nreturn [outputMsgs];",
+        "outputs": 1,
+        "noerr": 0,
+        "initialize": "",
+        "finalize": "",
+        "libs": [],
+        "x": 570,
+        "y": 80,
+        "wires": [["969cb4bae1c4c259"]]
+    }, {
+        "id": "ee5f8710c0a6b568",
+        "type": "comment",
+        "z": "f6ebb8e404985534",
+        "name": "splitting the JSON Object",
+        "info": "The ticker is split into individual json objects and passed on as messages ... ",
+        "x": 610,
+        "y": 40,
+        "wires": []
+    }, {
+        "id": "97961487e6cdb936",
+        "type": "http request",
+        "z": "f6ebb8e404985534",
+        "name": "QuestDB Query",
+        "method": "GET",
+        "ret": "txt",
+        "paytoqs": "query",
+        "url": "http://192.168.0.200:9000/exec",
+        "tls": "",
+        "persist": false,
+        "proxy": "",
+        "insecureHTTPParser": false,
+        "authType": "",
+        "senderr": false,
+        "headers": [],
+        "x": 860,
+        "y": 220,
+        "wires": [["d8824452cc11c4e7"]]
+    }, {
+        "id": "969cb4bae1c4c259",
+        "type": "function",
+        "z": "f6ebb8e404985534",
+        "name": "BuildSQL",
+        "func": "/*\ntimestamp: \"1671032171\"\nopen: \"3.82\"\nhigh: \"3.88\"\nlow: \"3.75\"\nlast: \"3.82\"\nvolume: \"3337.54\"\nvwap: \"3.82\"\nbid: \"3.81\"\nask: \"3.83\"\nopen_24: \"3.88\"\npercent_change_24: \"-1.55\"\npair: \"APE/EUR\"\n\n\n   \nVALUES(\n    '2021-10-05T11:31:35.878Z',\n    'AAPL',\n    255,\n    123.33,\n    'B');\n\n */\nvar q = {};\nvar symbol = msg.payload.pair.replace(\"/\", \"_\");\n\n// q[\"query\"] = \"select temp_c, humidity, timestamp from iot where timestamp > (systimestamp() - 5000000)\"\nq[\"query\"] = \"INSERT INTO  bs_ticker_\" + symbol + \n    \" VALUES(\" + msg.payload.open + \",\" + \n    \"\" + msg.payload.high + \",\" +\n    \"\" + msg.payload.low + \",\" +\n    \"\" + msg.payload.last + \",\" +\n    \"\" + msg.payload.volume + \",\" +\n    \"\" + msg.payload.bid + \",\" +\n    \"\" + msg.payload.ask + \",\" +\n    \"\" + msg.payload.open_24 + \",\" +\n    \"\" + msg.payload.percent_change_24 + \",\" +\n    \"\" + msg.timestamp*1000 + \");\";\nmsg.payload = q\nreturn msg;",
+        "outputs": 1,
+        "noerr": 0,
+        "initialize": "",
+        "finalize": "",
+        "libs": [],
+        "x": 580,
+        "y": 120,
+        "wires": [["97961487e6cdb936"]]
+    }, {
+        "id": "15d2128a7f055492",
+        "type": "inject",
+        "z": "f6ebb8e404985534",
+        "name": "On deploy",
+        "props": [{
+                "p": "payload"
+            }, {
+                "p": "topic",
+                "vt": "str"
+            }, {
+                "p": "timestamp",
+                "v": "",
+                "vt": "date"
+            }
+        ],
+        "repeat": "",
+        "crontab": "",
+        "once": true,
+        "onceDelay": 0.1,
+        "topic": "Bitstamp Ticker",
+        "payload": "",
+        "payloadType": "date",
+        "x": 130,
+        "y": 240,
+        "wires": [["38d461c4972558c8"]]
+    }, {
+        "id": "38d461c4972558c8",
+        "type": "http request",
+        "z": "f6ebb8e404985534",
+        "name": "",
+        "method": "GET",
+        "ret": "obj",
+        "paytoqs": "ignore",
+        "url": "https://www.bitstamp.net/api/v2/ticker/",
+        "tls": "",
+        "persist": false,
+        "proxy": "",
+        "insecureHTTPParser": false,
+        "authType": "",
+        "senderr": false,
+        "headers": [],
+        "x": 390,
+        "y": 240,
+        "wires": [["d49e767fdb202e20"]]
+    }, {
+        "id": "d49e767fdb202e20",
+        "type": "function",
+        "z": "f6ebb8e404985534",
+        "name": "splitTickers",
+        "func": "var outputMsgs = [];\n\nmsg.payload.forEach(myFunction);\n\nfunction myFunction(value) {\n    outputMsgs.push({ payload: value, timestamp: msg.timestamp });\n}\n\nreturn [outputMsgs];",
+        "outputs": 1,
+        "noerr": 0,
+        "initialize": "",
+        "finalize": "",
+        "libs": [],
+        "x": 570,
+        "y": 240,
+        "wires": [["b8345a0484d5c611"]]
+    }, {
+        "id": "b8345a0484d5c611",
+        "type": "function",
+        "z": "f6ebb8e404985534",
+        "name": "BuildSQL-CreateTables",
+        "func": "/*\ntimestamp: \"1671032171\"\nopen: \"3.82\"\nhigh: \"3.88\"\nlow: \"3.75\"\nlast: \"3.82\"\nvolume: \"3337.54\"\nvwap: \"3.82\"\nbid: \"3.81\"\nask: \"3.83\"\nopen_24: \"3.88\"\npercent_change_24: \"-1.55\"\npair: \"APE/EUR\"\n\nCREATE TABLE my_table(symb SYMBOL, price DOUBLE, ts TIMESTAMP, s STRING)\n  timestamp(ts);\n */\n\nvar q = {};\n// q[\"query\"] = \"select temp_c, humidity, timestamp from iot where timestamp > (systimestamp() - 5000000)\"\nvar symbol = msg.payload.pair.replace(\"/\", \"_\");\nq[\"query\"] = \n    \"CREATE TABLE IF NOT EXISTS bs_ticker_\" + symbol + \"(open DOUBLE, high DOUBLE, low DOUBLE, last DOUBLE, volume DOUBLE, bid DOUBLE, ask DOUBLE, open_24 DOUBLE, percent_change_24 DOUBLE, ts TIMESTAMP)timestamp(ts);\";\nmsg.payload = q\nreturn msg;",
+        "outputs": 1,
+        "noerr": 0,
+        "initialize": "",
+        "finalize": "",
+        "libs": [],
+        "x": 530,
+        "y": 280,
+        "wires": [["97961487e6cdb936"]]
+    }, {
+        "id": "067fb692727bf8c9",
+        "type": "catch",
+        "z": "f6ebb8e404985534",
+        "name": "Catch All",
+        "scope": null,
+        "uncaught": true,
+        "x": 840,
+        "y": 80,
+        "wires": [["f38c603f653186cb"]]
+    }, {
+        "id": "650da83460d687c3",
+        "type": "function",
+        "z": "f6ebb8e404985534",
+        "name": "BuildSQL-DropTables",
+        "func": "/*\ntimestamp: \"1671032171\"\nopen: \"3.82\"\nhigh: \"3.88\"\nlow: \"3.75\"\nlast: \"3.82\"\nvolume: \"3337.54\"\nvwap: \"3.82\"\nbid: \"3.81\"\nask: \"3.83\"\nopen_24: \"3.88\"\npercent_change_24: \"-1.55\"\npair: \"APE/EUR\"\n\nCREATE TABLE my_table(symb SYMBOL, price DOUBLE, ts TIMESTAMP, s STRING)\n  timestamp(ts);\n */\n\nvar q = {};\n// q[\"query\"] = \"select temp_c, humidity, timestamp from iot where timestamp > (systimestamp() - 5000000)\"\nvar symbol = msg.payload.pair.replace(\"/\", \"_\");\nq[\"query\"] = \n    \"DROP TABLE bs_ticker_\" + symbol + \";\";\nmsg.payload = q\nreturn msg;",
+        "outputs": 1,
+        "noerr": 0,
+        "initialize": "",
+        "finalize": "",
+        "libs": [],
+        "x": 540,
+        "y": 380,
+        "wires": [["97961487e6cdb936"]]
+    }, {
+        "id": "78f1d48e24430395",
+        "type": "inject",
+        "z": "f6ebb8e404985534",
+        "name": "When you click",
+        "props": [{
+                "p": "payload"
+            }, {
+                "p": "topic",
+                "vt": "str"
+            }, {
+                "p": "timestamp",
+                "v": "",
+                "vt": "date"
+            }
+        ],
+        "repeat": "",
+        "crontab": "",
+        "once": false,
+        "onceDelay": 0.1,
+        "topic": "Bitstamp Ticker",
+        "payload": "",
+        "payloadType": "date",
+        "x": 140,
+        "y": 340,
+        "wires": [["1841c9fc5cf66ce2"]]
+    }, {
+        "id": "1841c9fc5cf66ce2",
+        "type": "http request",
+        "z": "f6ebb8e404985534",
+        "name": "",
+        "method": "GET",
+        "ret": "obj",
+        "paytoqs": "ignore",
+        "url": "https://www.bitstamp.net/api/v2/ticker/",
+        "tls": "",
+        "persist": false,
+        "proxy": "",
+        "insecureHTTPParser": false,
+        "authType": "",
+        "senderr": false,
+        "headers": [],
+        "x": 390,
+        "y": 340,
+        "wires": [["11038880c4bdabe5"]]
+    }, {
+        "id": "11038880c4bdabe5",
+        "type": "function",
+        "z": "f6ebb8e404985534",
+        "name": "splitTickers",
+        "func": "var outputMsgs = [];\n\nmsg.payload.forEach(myFunction);\n\nfunction myFunction(value) {\n    outputMsgs.push({ payload: value, timestamp: msg.timestamp });\n}\n\nreturn [outputMsgs];",
+        "outputs": 1,
+        "noerr": 0,
+        "initialize": "",
+        "finalize": "",
+        "libs": [],
+        "x": 570,
+        "y": 340,
+        "wires": [["650da83460d687c3"]]
+    }, {
+        "id": "e2b640df5c6bd091",
+        "type": "comment",
+        "z": "f6ebb8e404985534",
+        "name": "The Loop",
+        "info": "Here we get the ticker everz minute and store the results in the db",
+        "x": 100,
+        "y": 40,
+        "wires": []
+    }, {
+        "id": "b2b559311fe4e399",
+        "type": "comment",
+        "z": "f6ebb8e404985534",
+        "name": "Create Tables - If not exist",
+        "info": "Here we get the ticker and create a table for each pair",
+        "x": 150,
+        "y": 200,
+        "wires": []
+    }, {
+        "id": "8d9caa02581087b1",
+        "type": "comment",
+        "z": "f6ebb8e404985534",
+        "name": "Drop Tables - This will delete all data!",
+        "info": "Here we get the ticker and drop the table for each pair",
+        "x": 190,
+        "y": 300,
+        "wires": []
+    }, {
+        "id": "f38c603f653186cb",
+        "type": "debug",
+        "z": "f6ebb8e404985534",
+        "name": "Errors",
+        "active": true,
+        "tosidebar": true,
+        "console": false,
+        "tostatus": false,
+        "complete": "payload",
+        "targetType": "msg",
+        "statusVal": "",
+        "statusType": "auto",
+        "x": 850,
+        "y": 120,
+        "wires": []
+    }
+]