pnl, tax, transactions

2025-03-01 15:28:50 +01:00 · 2025-02-28 17:22:42 +01:00 · 2025-02-27 10:44:55 +01:00 · 2025-02-22 16:49:50 +01:00 · 2024-12-25 13:44:11 +01:00 · 2024-12-25 13:31:36 +01:00
33 changed files with 1755 additions and 776 deletions
--- a/README.md
+++ b/README.md
@@ -13,7 +13,7 @@ It is still under development and is currently only partially functional.

 ## Copyright / License

-Copyright (c) 2022-2024 Oliver Bohlen (aka olli)
+Copyright (c) 2022-2024 Oliver Bohlen (aka olli/egabosh)

 The software provided here is called dabo (crypto bot)

@@ -25,11 +25,14 @@ You should have received a copy of the GNU General Public License along with dab

 ## Data sources
 Various data sources such as finance.yahoo.com and crypto exchanges available via ccxt are used. Please check whether this is legal in your region before use.
- query1.finance.yahoo.com (economic data,...)
- api.coinmarketcap.com (crypto data)
- 30rates.com (forecast)
- fapi.binance.com (OpenInterest,...)
- api.alternative.me (Fear and Greed)
+- https://query1.finance.yahoo.com (economic data,...)
+- https://api.coinmarketcap.com (crypto data)
+- https://api.bls.gov (CPI, unemployment rate)
+- https://fred.stlouisfed.org (fed funds rate)
+- https://30rates.com (forecast)
+- https://fapi.binance.com (OpenInterest,...)
+- https://api.alternative.me (Fear and Greed)
+- https://production.dataviz.cnn.io (Fear and Greed CNN)
 - https://unpkg.com/lightweight-charts/dist/lightweight-charts.standalone.production.js (TradingView Lightweitgt Charts)
 - ...

@@ -38,7 +41,7 @@ Various data sources such as finance.yahoo.com and crypto exchanges available vi
 - TradingView Lightweitgt Charts (https://www.tradingview.com/lightweight-charts/ | https://github.com/tradingview/lightweight-charts)
 - bash, python
 - several default linux programs like bc, wget,...
- gaboshlib (https://gitea.ds9.dedyn.io/olli/gaboshlib)
+- gaboshlib (https://github.com/egabosh/gaboshlib)
 - ...

 ## Objective
@@ -55,7 +58,7 @@ https://memory-alpha.fandom.com/wiki/Dabo
 I thought this fits quite well to the cryptotrading world and that's why I chose this name ;-)

 ## Structure
-The Bot is splitted in the following parts:
+The Bot is splitted in the following parts/containers:

 - dabo-bot: Basic Bot with buy and sell decisions based on self-definable strategies
 - dabo-symbols_ticker: Ticker for current symbols and prices
@@ -100,7 +103,11 @@ OHLCV = Open, High, Low, Close and Volume of a time unit
 - self-calculated significant levels (support/resist)
 ### Dabo Market Data
 - Yahoo Finance
- CoinMarketCyp
+- CoinMarketCap
+- BLS.gov
+- alternative.me
+- CNN
+- Fed
 ### Dabo Orders
 ### Dabo Transaction History
 - Support of additional Exchnages/Brokers JustTrade (CSV-Import) and Bitpanda (API+CSV-Import)
@@ -119,7 +126,7 @@ Originally this project was supposed to be a simple script to monitor prices of
 Finally, it's a hobby project and I have to see how and when I can find time for it, because there also has to be time for family, friends, work and other hobbies.
 If there is someone who would like to rewrite this bot in e.g. Python, I would be happy to support them as best I can with this task. Just let me know.

-## How to use/install (basic Linux knowledge required!)
+## How to install (basic Linux knowledge required!)

 Should run on every system with docker.

@@ -128,58 +135,79 @@ Tested and running with Debian 12 (Bookworm).
 https://www.debian.org/download
 https://www.raspberrypi.com/software/operating-systems/

-### 2: Run Ansible Playbooks
-On a clean Debian 12 system ypu can run my Ansible Playbooks to use the same environment the bot is developed and running.
+### 2: Way 1: Istall via anssible playbooks
+On a fresh Debian 12 system you can run my Ansible Playbooks to use the same environment the bot is developed and running.
 Please have a look what exactly the playbooks are doing if you are unsure.

+If the web interface (or matrix/turn) should be accessible via the internet, the server must be accessible on port 80/tcp (acme/letsencrypt) and 443/tcp.
+For private/home internet access, this may require port forwarding on your router and registration with a dyndns (DDNS) service. The DDNS service should allow subdomains.
+
+When using my playbooks and the services should be available to the internet, the hostname must correspond to the ddns name. 
+Here is an example for the dynu-DDNS service:
+- DDNS name: myhost.mywire.com
+- dabo-bot-name: dabo.myhost.mywire.com
+- matrix-name: matrix.myhost.mywire.com
+
+You also can use IPv6 if your ISP and router supports this and if you have multiple servers/services on Ports (80/443) but only one IPv4 address to the internet.
+
 #### 2.1 Download basic install script
 ```
-wget https://gitea.ds9.dedyn.io/olli/debian.ansible.basics/raw/branch/main/install.sh
+wget https://raw.githubusercontent.com/egabosh/linux-setups/refs/heads/main/debian/install.sh -O install.sh
 ```
 #### 2.2 define Playbooks
- debian.ansible.basics (https://gitea.ds9.dedyn.io/olli/debian.ansible.basics) - Basic Debian configuration
- Optional: debian.ansible.firewall (https://gitea.ds9.dedyn.io/olli/debian.ansible.firewall) - Firewall for the server based on ufw
- Optional: debian.ansible.runchecks (https://gitea.ds9.dedyn.io/olli/debian.ansible.runchecks) - System checks and notification
- Optional: debian.ansible.backup (https://gitea.ds9.dedyn.io/olli/debian.ansible.backup/src/branch/main/backup.yml) - Backup framework
- Optional: debian.ansible.autoupdate (https://gitea.ds9.dedyn.io/olli/debian.ansible.autoupdate) - Automatic System Updates
- debian.ansible.docker (https://gitea.ds9.dedyn.io/olli/debian.ansible.docker/src/branch/main/docker.yml) - Docker Installation
- debian.ansible.traefik.server (https://gitea.ds9.dedyn.io/olli/debian.ansible.traefik.server) - Traefik Reverse Proxy for Web UI and Letsencrypt Certs
- Optional: debian.ansible.turn.server (https://gitea.ds9.dedyn.io/olli/debian.ansible.turn.server) - Turn Server fpr Audio/Video conferences in Matrix
- Optional: debian.ansible.matrix.server (https://gitea.ds9.dedyn.io/olli/debian.ansible.matrix.server) - Notifications with own Martix Server
- dabo: The Bot itself
+- debian/basics/basics.yml (https://github.com/egabosh/linux-setups/tree/main/debian/basics) - Basic Debian configuration
+- Optional: debian/firewall/firewall.yml (https://github.com/egabosh/linux-setups/tree/main/debian/firewall) - Firewall for the server based on ufw
+- Optional: debian/runchecks/runchecks.yml (https://github.com/egabosh/linux-setups/tree/main/debian/runchecks) - System checks and notification
+- Optional: debian/backup/backup.yml (https://github.com/egabosh/linux-setups/tree/main/debian/backup) - Backup framework
+- Optional: debian/autoupdate/autoupdate.yml (https://github.com/egabosh/linux-setups/tree/main/debian/autoupdate) - Automatic System Updates
+- debian/docker/docker.yml (https://github.com/egabosh/linux-setups/tree/main/debian/docker) - Docker Installation
+- debian/traefik.server/traefik.yml (https://github.com/egabosh/linux-setups/tree/main/debian/traefik.server) - Traefik Reverse Proxy for Web UI and Letsencrypt Certs
+- Optional: debian/turn.server/turn.yml (https://github.com/egabosh/linux-setups/tree/main/debian/turn.server) - Turn Server fpr Audio/Video conferences in Matrix
+- Optional: debian/matrix.server/matrix.yml (https://github.com/egabosh/linux-setups/tree/main/debian/matrix.server) - Notifications with own Martix Server
+- https://github.com/egabosh/dabo/raw/refs/heads/main/dabo-ansible.yml: The Bot itself

 for example:
 ```
-PLAYBOOKS="debian.ansible.basics
- debian.ansible.firewall
- debian.ansible.runchecks
- debian.ansible.backup
- debian.ansible.autoupdate
- debian.ansible.docker
- debian.ansible.traefik.server
- debian.ansible.turn.server
- debian.ansible.matrix.server
- dabo"
+PLAYBOOKS="debian/basics/basics.yml
+ debian/firewall/firewall.yml
+ debian/runchecks/runchecks.yml
+ debian/backup/backup.yml
+ debian/autoupdate/autoupdate.yml
+ debian/docker/docker.yml 
+ debian/traefik.server/traefik.yml 
+ https://github.com/egabosh/dabo/raw/refs/heads/main/dabo-ansible.yml"
 export PLAYBOOKS
 ```
 #### 2.3 Install ansible and run Playbooks
+If you run this as user and not as root, the script will install sudo and enable the user to execute commands as root via sudo. To do this, the root password is requested (several times).
 ```
 bash install.sh
 ```
-### Download
+
+### 3. Manual installation without ansible playbooks
+Not necessary if you use the dabo Playbook
+
+#### Install docker
+See: https://docs.docker.com/engine/install/debian/
+
+You can also use other containerizations like podman or kubernetes. But here I stick with docker
+
+#### Download
 Not necessary if you use the dabo Playbook
 ```
-git clone https://gitea.ds9.dedyn.io/olli/dabo.git
+git clone https://github.com/egabosh/dabo.git
 cd dabo
 ```

-### Build container
+#### Build container
 Not necessary if you use the dabo Playbook
 ```
 docker -l warn compose --ansi never build --progress=plain --pull --no-cache --force-rm
 ```

-### 3. Configuration
+### 3. Make Webinterface available
+Not necessary if you use the dabo Playbook with traefik and letsencrypt
+
 Edit docker-compose.yml or create docker-compose.override.yml to fit yout needs e.g. domain and network settings or basic auth, e.g. for traefik and letsencrypt:
 ```
 echo '
@@ -221,8 +249,17 @@ networks:
 ' >docker-compose.override.yml
 ```

+If you use the ansible Playbook but don't want to use traefik/letsencrypt or availability over the internet you can add the following to `docker-compose.override.yml` below "# END ANSIBLE MANAGED BLOCK".
+ATTENTION: ACCESS IS UNENCRYPTED!!!
+```
+    ports:
+      - 8080:80
+```
+You have to restart the container(s) to apply changes. See operational commands.

-Optional: If you use matrix/matrix-commander (https://gitea.ds9.dedyn.io/olli/debian.ansible.matrix.server) and want do receive Matrix-Messages from the bot you can create an SSH-Key to allow sending Matrix-Messages e.g.:
+### 4. Optional: Matrix connection
+
+Optional: If you use matrix/matrix-commander (https://github.com/egabosh/linux-setups/tree/main/debian/matrix.server) and want do receive Matrix-Messages from the bot you can create an SSH-Key to allow sending Matrix-Messages e.g.:
 Automatically done by playbooks.
 ```
 mkdir -p home/.ssh
@@ -232,14 +269,12 @@ cat home/.ssh/id_ed25519.pub
 ```
 and add Key on your matrix-Server to the authorized_keys of the matrix-User

-
+### 5. Add Exchange
 Create Secrets file for your API Key(s)

 - file: dabo/.CCXT-ID-secrets
-
 CCXI-ID see: https://github.com/ccxt/ccxt

-
 Examples:
 ```
 # for Phemex 
@@ -255,53 +290,13 @@ chmod 400 dabo/.binance-secrets

 Create Config
 Especially set URL, STOCK_EXCHANGE, FEE, CURRENCY,... to fit your needs.
+If you want to use the a testnet of an exchnage if available use TESTNET=true which is the default
 ```
 vim dabo-bot.conf
 ```
 Defaults in dabo/dabo-bot.conf

-
-### 4. Prepare/Create a stretegy
-
-IMPORTANT!!! 
-
-THE DEFAULT STRATEGY MAY NOT FIT YOUR NEEDS OR WORK PROPERLY. SO YOU CAN LOOSE ALL YOUR MONEY!!! USE ON YOUR OWN RISK!!!
-
-TEST YOUR OWN STRATEGY COMPREHENSIVELY AND OVER A LOGNER PERIOD OF TIME WITH analyze.sh!!! USE ON YOUR OWN RISK!!!
-
-Strategie files can be put in the "strategies"-directory the defaults 
-
-There is an example for a buy and a sell strategy file (deactivated by "return 0" in the forst line):
-```
-ls strategies/buy.example.conf
-ls strategies/sell.example.conf
-```
-Aditional strategies can be created with Name
-```
-strategies/buy.<name>.conf
-strategies/sell.<name>.conf
-```
-e.g named "mannover-sulu-1" for buy strategy and "command-kirk-3" for sell strategy
-```
-strategies/buy.mannover-sulu-1.conf
-strategies/sell.command-kirk-3.conf
-```
-
-### Optional: Create individual watch-assets.csv
-```
-cp dabo/watch-assets.csv watch-assets.csv
-```
-Optional:
-You can edit this file if you want do generate warnings or track your asstes/trades.
-```
-nano watch-assets.csv
-```
-
-### Set Rights
-Set Rights (UID 10000 for non-root-User in running container):
-```
-chown -R 10000:10000 dabo data home strategies dabo-bot.conf watch-assets.csv
-```
+## How to use

 ### Operational commands
 Run/Restart:
@@ -310,7 +305,7 @@ docker compose down   # if an old instance is running
 docker compose up -d 
 ```

-Check
+List and state of containers:
 ```
 docker compose ps
 ```
@@ -319,29 +314,76 @@ Logs/Output:
 ```
 docker compose logs -f
 ```
+Logs/Oputput of a specific container for example dabo-bot:
+```
+docker compose logs -f dabo-bot
+```

-Update:
+
+### Update
 Not necessary if you use the playbooks
 ```
-# Optinal: Remove local data
+# Optional: Remove local data
 git reset --hard HEAD^   # Remove local commits
 git clean -fd            # Remove local uncommited files

 # Update and restart
-git pull https://gitea.ds9.dedyn.io/olli/dabo.git main -f
+git pull https://github.com/egabosh/dabo.git main -f
 docker compose down 
 docker compose up -d
 ```

-## Strategies
+### Strategies
+
+IMPORTANT!!!
+
+THE EXAMPLE STRATEGIES MAY NOT FIT YOUR NEEDS OR WORK PROPERLY. SO YOU CAN LOOSE ALL YOUR MONEY!!! USE ON YOUR OWN RISK!!!
+
+TEST YOUR OWN STRATEGY COMPREHENSIVELY AND OVER A LOGNER PERIOD OF TIME BEST RISK-FREE IN A TESTNET!!! USE ON YOUR OWN RISK!!!
+
+Strategies are needed for the bot to trade.
+
+Strategies are located in die stretegies subdir.

 You can put your own code into the strategies it will be sourced by the bot.
+If you want, you can also use other programming languages or binary code as a strategy and simply start it using your own strategy.

-You can use available variables to read (and set) values.
+#### Example strategies
+There are examples for strategy files (deactivated by "return 0" in the beginning):
+```
+cat strategies/example.strategy.sh
+cat strategies/example_manage_positions.strategy.sh
+```
+"example.strategy.sh" creates a score by defined tests and opens a long or short order with stoploss and takeprofit if the score has a defined value.

-### Variables with current market values
+"example_manage_positions.strategy.sh" watches open positions and switches the stoploss into profit if position is in profit to secure it.

-#### Large associative arrays v and vr (reverse)
+You can use them and change them to fit your needs. To avoid resets on updates copy them to your own strategies for example:
+
+```
+cp -p strategies/example.strategy.sh                   strategies/my-own-trading.strategy.sh
+cp -p strategies/example_manage_positions.strategy.sh  strategies/my-own-position.strategy.sh
+```
+
+#### Own strategies
+Aditional strategies can be created with naming convention *.strategy.sh
+```
+strategies/my-new.strategy.sh
+strategies/yet-another-new.strategy.sh
+```
+
+Strategy files should have specific rights - must be readable by the bot:
+```
+chown -R 10000:10000 strategies
+chmod -R 640 strategies
+``` 
+
+#### Variables during runtime of the strategies
+
+You can use available variables/arrays during runtime to read (and set) values.
+This arrays are available to runtime in the strategies
+
+##### Large associative arrays v and vr (reverse)

 ${v[SYMBOL_TIMEFRAME_ITEM_NUMBER]}

@@ -357,12 +399,13 @@ ${v[ECONOMY_SP500_rsi14_0]}
 ${v[ETHUSDT_1w_ema200_0]}
 ${v[ETHUSDT_1w_macd_histogram_signal_1]}
 ${v[SOLUSDT_levels_1d]}
-$v[$[SOLUSDT_levels_1d_next_up]}
-$v[$[ETHUSDT_levels_1w_next_down]}
+${v[$[SOLUSDT_levels_1d_next_up]}
+${v[$[ETHUSDT_levels_1w_next_down]}
 ```
-You can find a complete list of available values in the file `data/botdata/values` whic is creates in the runtime of the bot.
-An example you can find in example-values.
-#### Current price from exchange 
+You can find a complete list of available values in the file `data/botdata/values` which is created in the runtime of the bot.
+An long example list you can find in example-values. https://github.com/egabosh/dabo/blob/main/example-values
+
+##### Current price from exchange 

 ${f_tickers_array[SYMBOL]}
 ```
@@ -370,18 +413,61 @@ ${f_tickers_array[SOLUSDT]}
 ${f_tickers_array[ETH${CURRENCY}]}
 ```

+##### Open Orders
+
+```
+${o[ETHUSDT_present]}=sl_close_long tp_close_long
+${o[ETHUSDT_sl_close_long_amount]}=0
+${o[ETHUSDT_sl_close_long_entry_price]}=null
+${o[ETHUSDT_sl_close_long_id]}=36b2f404-0b20-4806-bdcc-0dad0daf57e9
+${o[ETHUSDT_sl_close_long_side]}=sell
+${o[ETHUSDT_sl_close_long_stoplossprice]}=0
+${o[ETHUSDT_sl_close_long_stopprice]}=3305.98
+${o[ETHUSDT_sl_close_long_takeprofitprice]}=0
+${o[ETHUSDT_sl_close_long_type]}=Stop
+${o[ETHUSDT_tp_close_long_amount]}=0
+${o[ETHUSDT_tp_close_long_entry_price]}=null
+${o[ETHUSDT_tp_close_long_id]}=850dd169-7387-4be3-ac68-bfce73cfa47d
+${o[ETHUSDT_tp_close_long_side]}=sell
+${o[ETHUSDT_tp_close_long_stoplossprice]}=0
+${o[ETHUSDT_tp_close_long_stopprice]}=3710.04
+${o[ETHUSDT_tp_close_long_takeprofitprice]}=0
+${o[ETHUSDT_tp_close_long_type]}=MarketIfTouched
+```
+
+You can find a complete list of available values in the file `data/botdata/values-orders` which is created in the runtime of the bot.
+
+
+##### Open Positions
+
+```
+${p[ETHUSDT_currency_amount]}=9509.25
+${p[ETHUSDT_current_price]}=3573.79
+${p[ETHUSDT_entry_price]}=3673.31
+${p[ETHUSDT_leverage]}=5
+${p[ETHUSDT_liquidation_price]}=2953.43
+${p[ETHUSDT_pnl]}=-1288.50
+${p[ETHUSDT_pnl_percentage]}=-13.55
+${p[ETHUSDT_side]}=long
+${p[ETHUSDT_stoploss_price]}=3305.98
+${p[ETHUSDT_takeprofit_price]}=3710.04
+```
+
+You can find a complete list of available values in the file `data/botdata/values-positions` which is created in the runtime of the bot.
+
+
 ## Support/Community
-New Telegram group for the dabo comunity. 
+New Telegram group for the dabo community. 
 https://t.me/dabobotcrypto


-## Future ideas/featrues and todosa
- Partial StopLoss and TakeProfit
- Fibonacci Levels
- Fear and Greed Indicator (api.alternative.me)
+## Future ideas/featrues and todos
+- PSR Indicator https://de.tradingview.com/script/w4U2xUN7-Pivot-Support-Resistance/ 
+- public order data if there is such a thing. At what price do people want to enter (limit order)? Where do they want to exit (TakePofit/StopLoss)? Maybe free Binance data helps (https://developers.binance.com/docs/derivatives/usds-margined-futures/market-data/rest-api/Order-Book)?  https://www.coinglass.com/de/mergev2/BTC-USDT
+- AI implementation. What data can be prepared for training?
+- example Grid trading strategy (user specific)
+- Fibonacci Levels (which timeframe(s)?)
 - Forecasts
- US Government data like unemployment rates https://www.bls.gov/developers/api_unix.htm#unix1
- Example strategie's
 - Chart improvements
 - Volumeindicator and for example RSI on volume values
 - Support for decentralized exchanges like uniswap
@@ -390,6 +476,5 @@ https://t.me/dabobotcrypto
 - Emergency stop if balance falls below defined value
 - Analysis tool for collected historical values to try out buy or sell conditions based on them 
 - Consideration of trading and funding fees
- market-performance?
- OpenInterest (fapi.binance.com)
+- Liquidation Heatmap (https://www.coinglass.com/pro/futures/LiquidationHeatMap)

--- a/dabo-ansible.yml
+++ b/dabo-ansible.yml
@@ -13,14 +13,14 @@

    - name: Git checkout
      ansible.builtin.git:
-        repo: 'https://gitea.ds9.dedyn.io/olli/dabo.git'
+        repo: 'https://github.com/egabosh/dabo.git'
        dest: /home/docker/dabo.{{inventory_hostname}}
        force: true
      notify: Restart dabo

-    - name: /home/docker/dabo.{{inventory_hostname}}/genpw.sh (generate Random PW)
+    - name: /home/docker/dabo.{{inventory_hostname}}/initiate.sh
      blockinfile:
-        path: /home/docker/dabo.{{inventory_hostname}}/genpw.sh
+        path: /home/docker/dabo.{{inventory_hostname}}/initiate.sh
        create: yes
        mode: 0550
        owner: root
@@ -29,41 +29,22 @@
        block: |
          cd /home/docker/dabo.{{inventory_hostname}}

-          if [ -f env ]
-          then
-            . ./env
-            echo "${WEBPASSWDCRYPT}"
-          else
-            webpassword=$(pwgen -s 32 1)
-            webuser=bot
-            webpasswordcrypted=$(htpasswd -nb ${webuser} ${webpassword} | sed -e 's/\\$/\\$\\$/g')
- 
-            echo "WEBUSER=${webuser}
-          WEBPASSWD='${webpassword}'
-          WEBPASSWDCRYPT='${webpasswordcrypted}'
-          " >env
-
-            chmod 440 env
-            chown root:docker env
-            echo "${webpasswordcrypted}"
-          fi
-
          if ! [ -d home/.ssh ]
          then
            mkdir -p home/.ssh
            ssh-keygen -f home/.ssh/id_ed25519 -N "" -t ed25519 >/dev/null
            chmod 700 home/.ssh
          fi
- 
-          #[ -f watch-assets.csv ] || cp dabo/watch-assets.csv watch-assets.csv
-          chown -R 10000:10000 dabo data home strategies dabo-bot.conf #watch-assets.csv
+          mkdir -p data/botdata strategies 
+          chown -R 10000:10000 dabo data home strategies dabo-bot.conf

        backup: yes
        validate: /bin/bash -n %s
+      notify: run initiate.sh

-    - name: /home/docker/dabo.{{inventory_hostname}}/genpw.sh shebang
+    - name: /home/docker/dabo.{{inventory_hostname}}/initiate.sh shebang
      lineinfile:
-        path: /home/docker/dabo.{{inventory_hostname}}/genpw.sh
+        path: /home/docker/dabo.{{inventory_hostname}}/initiate.sh
        insertbefore: BOF
        line: "#!/bin/bash -e"

@@ -84,10 +65,10 @@
      - Restart dabo


-    - name: Get crypted PW
-      shell: bash /home/docker/dabo.{{inventory_hostname}}/genpw.sh
-      register: cryptpw
-      changed_when: false
+    #- name: Get crypted PW
+    #  shell: bash /home/docker/dabo.{{inventory_hostname}}/initiate.sh
+    #  register: cryptpw
+    #  changed_when: false

    - name: /home/docker/dabo.{{inventory_hostname}}/docker-compose.override.yml Container Configuration
      blockinfile:
@@ -98,6 +79,14 @@
        group: docker
        marker: "# {mark} ANSIBLE MANAGED BLOCK"
        block: |
+          networks:
+            dabo-network:
+              driver: bridge
+              driver_opts:
+                com.docker.network.bridge.name: br-dabo
+            traefik:
+              external: true
+
          services:

            dabo-bot:
@@ -118,21 +107,11 @@
                # cert via letsencrypt
                - traefik.http.routers.dabo.tls.certresolver=letsencrypt
                # activate secHeaders@file
-                - traefik.http.routers.dabo.middlewares=secHeaders@file,dabo-basicauth
-                - traefik.http.middlewares.dabo-basicauth.basicauth.users={{ cryptpw.stdout }}
+                - traefik.http.routers.dabo.middlewares=secHeaders@file,default-basic-auth@file
                # Traefik network
                - traefik.docker.network=traefik
              networks:
                - traefik
-
-          networks:
-            dabo-network:
-              driver: bridge
-              driver_opts:
-                com.docker.network.bridge.name: br-dabo
-            traefik:
-              external: true          
-
        backup: yes
      notify: Restart dabo

@@ -142,11 +121,26 @@
        chdir: /home/docker/dabo.{{inventory_hostname}}
        creates: /home/docker/dabo.{{inventory_hostname}}/data/botdata/MARKET_PERFORMANCE

+    - name: autoupdate dabo-bot
+      blockinfile:
+        path: /usr/local/sbin/autoupdate.d/dabo.update
+        create: yes
+        mode: 0550
+        owner: root
+        group: root
+        marker: "# {mark} ANSIBLE MANAGED BLOCK"
+        block: |
+          g_echo_ok "Dabo update"
+          rm -f /tmp/dabo-ansible.yml
+          wget https://raw.githubusercontent.com/egabosh/dabo/refs/heads/main/dabo-ansible.yml -O /tmp/dabo-ansible.yml
+          ansible-playbook --connection=local --inventory $(hostname), --limit $(hostname) /tmp/dabo-ansible.yml
+        backup: yes
+        validate: /bin/bash -n %s
 
  handlers:

-    - name: run genpw.sh
-      ansible.builtin.shell: ./genpw.sh
+    - name: run initiate.sh
+      ansible.builtin.shell: ./initiate.sh
      args:
        chdir: /home/docker/dabo.{{inventory_hostname}}
      notify: Restart dabo
@@ -155,3 +149,4 @@
      ansible.builtin.shell: docker compose up -d --force-recreate
      args:
        chdir: /home/docker/dabo.{{inventory_hostname}}
+
--- a/dabo/calc-indicators-hist.sh
+++ b/dabo/calc-indicators-hist.sh
@@ -30,8 +30,9 @@ do
  # Reload Config
  . ../../dabo-bot.conf
  . ../../dabo-bot.override.conf
-  # get orders
+  # get all indicators first only latest until EMA800
+  get_indicators_all 810
  get_indicators_all
-  rm asset-histories/*.history.*.csv.indicators-calculating
+  rm -f asset-histories/*.history.*.csv.indicators-calculating
 done

--- a/dabo/calc-levels.sh
+++ b/dabo/calc-levels.sh
@@ -22,7 +22,7 @@
 . /dabo/dabo-prep.sh

 rm -f asset-histories/*.history.*.csv.levels-calculating
-sleep 12
+sleep 120

 while true
 do
@@ -33,6 +33,7 @@ do
  # get levels
  get_levels_all
  rm -f asset-histories/*.history.*.csv.levels-calculating
-  sleep 900
+  # recalc every 6 hours
+  sleep 21600
 done

--- a/dabo/dabo-bot.conf
+++ b/dabo/dabo-bot.conf
@@ -2,16 +2,18 @@

 # Webpage URL
 URL="mydabobot.mydomain"
+URL="dabo.$(hostname)"

 # The exchange we use for using the correct API (CCXI-ID see: https://github.com/ccxt/ccxt)
+# for Phemex it is "phemex" for example
 STOCK_EXCHANGE="NONE"

-# Interval in seconds - Should not be lower then 300
+# Interval in seconds - Should not be lower then 300 seconds (5m is the lowest timeframe
 INTERVAL="300"

 ## Currency used for trading
 CURRENCY="USDT"
-TRANSFER_CURRENCY="USD"
+TRANSFER_CURRENCY="EUR"

 # symbols that should be traded
 SYMBOLS="ETH SOL"
@@ -19,16 +21,12 @@ SYMBOLS="ETH SOL"
 ## Signal Group for Notifications
 NOTIFICATION_GROUP="Dabo-Bot"

-## Percent from balance per invest.
-# Overwritten by MIN_NOTIONAL+X% from stock if lower
-INVEST="5"
-
-# Stop all trading and sell everything if the complete balance shrinks under this value in ${CURRENCY}
-EMERGENCY_STOP="1000"
-
 # Leverage
-LEVERAGE=""
+LEVERAGE="2"

 # margin mode (isolated or cross)
 MARGIN_MODE="isolated"

+# testnet of exchange if available (true or false). false for trades with real money
+TESTNET=true
+
--- a/dabo/dabo-bot.sh
+++ b/dabo/dabo-bot.sh
@@ -71,42 +71,14 @@ do
  # Timestamp
  export f_timestamp=$(g_date_print)

-
-####### TODO -> Funktionen überarbeiten ############
-#
-#  # get minute interval for find -mmin (used by get_marketdata market_performance
-#  INTERVAL_MIN=$(echo "${INTERVAL}/60-1" | bc -l | sed -r 's/^(-?)\./\10./' | cut -d\. -f1)
-#  [ -z "${INTERVAL_MIN}" ] && INTERVAL_MIN=1
-#
-#  ### get general market data
-#  # Get coingecko data
-#  get_coingecko_data
-#
-#  # Get current MarketData
-#  get_marketdata
-#
-#  # Check the situation on the market
-#  if ! market_performance
-#  then
-#    f_market_performance=$(cat MARKET_PERFORMANCE_LATEST)
-#  fi
-####### TODO -> Funktionen überarbeiten ENDE ###########
-
-
-  ## watch some manual defined assets
-  #watch_assets
-  
  if [ "${STOCK_EXCHANGE}" = "NONE" ]
  then
    ## stop here if STOCK_EXCHANGE not present
    continue   
  fi

-  # Get current symbols
-  #[ ${FULL_LOOP} = 1 ] && get_symbols_ticker
- 
-  # Sell something?
-  #check_for_sell
+  # clean old data
+  [ ${FULL_LOOP} = 1 ] && find asset-histories -maxdepth 1 ! -type d ! -name "*.csv" ! -name "*.levels" ! -name "*.zones" !  -name "*.indicators-calculated" -mtime +1 -delete

  # Get current balance
  [ ${FULL_LOOP} = 1 ] && get_balance || continue
--- a/dabo/fetch-ohlcv-candles-indicators.sh
+++ b/dabo/fetch-ohlcv-candles-indicators.sh
@@ -31,16 +31,25 @@ do
  # Reload Config
  . ../../dabo-bot.conf
  . ../../dabo-bot.override.conf
-  # notify failed yahoo downloads
-  [ "$interval" = "1d" ] && cat FAILED_YAHOO/*USD_* FAILED_YAHOO/*ECONOMY* 2>/dev/null | notify.sh -s "Failed Yahoo downloads"
+  # notify failed downloads
+  if [ "$interval" = "1h" ] 
+  then
+    cat FAILED_*/* 2>/dev/null | notify.sh -s "Failed downloads"
+    mkdir -p REPORTED_FAILED
+    mv FAILED_*/* REPORTED_FAILED/ 2>/dev/null
+  fi
  # Timestamp
  export f_timestamp=$(g_date_print)
  # get candles and indicators
  get_ohlcv-candles $interval
+  [[ $interval != 1w ]] && get_marketdata_all $interval
  [ -n "$seconds" ] && sleeptime=$(( ( ($seconds - $(TZ=UTC printf "%(%s)T") % $seconds) % $seconds + 2 )))
  #[[ $interval = 4h ]] && sleeptime=??
-  [ "$interval" = "1d" ] && sleeptime=$(($(TZ=UTC date +%s -d "tomorrow 0:00") - $(date +%s) +2 ))
-  [ "$interval" = "1w" ] && sleeptime=$(($(TZ=UTC date +%s -d "next monday 0:00") - $(date +%s) +2 ))
+  if [ "$interval" = "1d" ] 
+  then
+    sleeptime=$(($(TZ=UTC date +%s -d "tomorrow 0:01") - $(date +%s) +2 ))
+  fi
+  [ "$interval" = "1w" ] && sleeptime=$(($(TZ=UTC date +%s -d "next monday 0:01") - $(date +%s) +2 ))
  g_echo_note "Waiting $sleeptime seconds until next run"
  sleep $sleeptime
 done
--- a/dabo/fetch-transaction-history.sh
+++ b/dabo/fetch-transaction-history.sh
@@ -24,8 +24,19 @@
 sleep 1800
 while true
 do
+  >ALL_TRANSACTIONS_OVERVIEW.csv.tmp
  g_echo_note "Next loop"
-  transactions_overview
+  get_bitpanda_api_transactions
+  get_justtrade_csv_transactions
+  get_onetrading_csv_transactions
+  get_phemex_csv_transactions
+  get_transactions
+  for transaction_csv in TRANSACTIONS-*.csv
+  do
+     calc_fifo_pnl "$transaction_csv"
+  done
+  mv ALL_TRANSACTIONS_OVERVIEW.csv.tmp ALL_TRANSACTIONS_OVERVIEW.csv
+  webpage_transactions
  sleep 3600
 done

--- a/dabo/functions/calc_ema.sh
+++ b/dabo/functions/calc_ema.sh
@@ -40,9 +40,6 @@ function calc_ema {
  # check if there is a column (i from loop through array)
  [ -z "$f_column" ] && return 3
  
-  # check for enough positions/values to calculate (enough values)
-  [ $f_position -ge $f_period ] || return 0
-
  # get ema column
  [ -z "$f_target_column" ] && local f_target_column="ema$f_period"

@@ -58,6 +55,12 @@ function calc_ema {
  local f_last_ema_position=$((f_position-1))
  local f_last_ema=${v_csv_array_associative[${f_target_column}_${f_last_ema_position}]}

+  # check for enough positions/values to calculate (enough values) if SMA needed
+  if [ -z "$f_last_ema" ]
+  then
+    [ $f_position -ge $f_period ] || return 5
+  fi
+
  # check if last EMA is given
  if [ -n "$f_last_ema" ]
  then
@@ -94,7 +97,6 @@ function calc_ema {
  fi
  v_csv_array_associative[${f_target_column}_${f_position}]=$g_calc_result
  f_ema=$g_calc_result
-
  return 0
 }

--- a/dabo/functions/calc_fifo_pnl.sh
+++ b/dabo/functions/calc_fifo_pnl.sh
@@ -0,0 +1,525 @@
+#!/bin/bash
+
+# Copyright (c) 2022-2024 olli
+#
+# This file is part of dabo (crypto bot).
+# 
+# dabo is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# dabo is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with dabo. If not, see <http://www.gnu.org/licenses/>.
+
+
+function calc_fifo_pnl {
+  # Initialize variables
+  local f_csv_file="$1"
+  local f_current_year=$(date +%Y)
+  declare -A f_holdings
+  local f_date f_action f_symbol f_crypto_amount f_fiat_currency f_fiat_amount f_exchange f_fee_currency f_fee_amount f_note f_fiat_amount_tax_currency
+
+  # Read CSV file line by line
+  while IFS=',' read -r f_date f_action f_symbol f_crypto_amount f_fiat_currency f_fiat_amount f_exchange f_fee_currency f_fee_amount f_note
+  do
+
+    ## Debug
+    #[ "$f_symbol" == "ETH" ] || continue
+    #[ "$f_note" == "short" ] && continue
+ 
+    # ignore stable coins 
+    [[ $f_symbol == USDT || $f_symbol == EUR ]] && continue
+
+    # Extract year from date
+    local f_year=${f_date:0:4}
+
+    ## Debug
+    #[ "$f_year" == "2024" ] || continue
+    #[ "$f_action" == "fundingfee" ] && continue
+
+    # add exchange 
+    [[ "$f_exchanges" != *"$f_exchange "* ]] && f_exchanges+="$f_exchange "
+
+    # prevent exponential numbers
+    g_num_exponential2normal $f_crypto_amount
+    f_crypto_amount=$g_num_exponential2normal_result
+    g_num_exponential2normal $f_fiat_amount
+    f_fiat_amount=$g_num_exponential2normal_result
+    g_num_exponential2normal $f_fee_amount
+    f_fee_amount=$g_num_exponential2normal_result
+
+    ## Debug
+    #echo "f_fiat_amount=$f_fiat_amount"
+
+    # convert f_fiat_currency/f_fiat_amount to TRANSFER_CURRENCY/f_fiat_amount_tax_currency if they are not equal
+    if ! [ "$f_fiat_currency" == "$TRANSFER_CURRENCY" ] && [ "$f_fiat_amount" != "0" ]
+    then
+      currency_converter $f_fiat_amount "$f_fiat_currency" $TRANSFER_CURRENCY "$f_date" >/dev/null
+      f_fiat_amount_tax_currency=$f_currency_converter_result
+    else
+      f_fiat_amount_tax_currency=$f_fiat_amount
+    fi
+    
+    ## Debug
+    #echo "f_fiat_amount_tax_currency=$f_fiat_amount_tax_currency"
+
+    # convert f_fee_currency/f_fee_amount to TRANSFER_CURRENCY/f_fiat_amount_tax_currency if present
+    if [ -n "$f_fee_amount" ]
+    then
+      currency_converter $f_fee_amount $f_fee_currency $TRANSFER_CURRENCY "$f_date" >/dev/null
+      f_fee_amount=$f_currency_converter_result
+      [[ $f_action == "buy" || $f_action == "leverage-buy" ]] && g_calc "$f_fiat_amount_tax_currency + $f_fee_amount"
+      [[ $f_action == "sell" || $f_action == "leverage-sell" || $f_action == "liquidation" ]] && g_calc "$f_fiat_amount_tax_currency - $f_fee_amount"
+      f_fiat_amount_tax_currency=$g_calc_result
+    fi
+
+    ## Debug
+    #echo "f_fiat_amount_tax_currency=$f_fiat_amount_tax_currency"
+
+    # no space in date (prevent problems mit $f_holdings)
+    f_date="${f_date/ /T}"
+    
+    # get current holfings for determining if this is a long or short trade (f_holdings_amount)
+    get_holdings_amount
+
+    ## Debug
+    #echo "f_crypto_amount=$f_crypto_amount"
+
+    # Process fundingfee action
+    if [[ $f_action == "fundingfee" ]]
+    then
+      process_fundingfee "$f_symbol" "$f_crypto_amount" "$f_fee_amount" "$f_date" "$f_year"
+    # Process buy actions
+    elif [[ $f_action == "buy" || $f_action == "leverage-buy" || $f_action == "reward-staking" || $f_action == "giveaway" || $f_action == "instant_trade_bonus" ]]
+    then
+      if g_num_is_higher_equal $f_holdings_amount 0
+      then
+        # long
+        ## Debug
+        #echo process_buy "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date"
+        process_buy "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date"
+      elif g_num_is_lower_equal "$f_holdings_amount" "-$f_crypto_amount"
+      then
+        # short
+        ## Debug
+        #echo process_sell "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date" "$f_year" short
+        process_sell "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date" "$f_year" short
+      else
+        # long+short (partial)
+        # calc long/short parts
+        g_percentage-diff "-$f_crypto_amount" "$f_holdings_amount"
+        g_calc "$f_fiat_amount_tax_currency/100*($g_percentage_diff_result+100)"
+        f_fiat_amount_tax_currency_long=$g_calc_result
+        g_calc "$f_fiat_amount_tax_currency/100*(($g_percentage_diff_result*-1))"
+        f_fiat_amount_tax_currency_short=$g_calc_result
+        g_calc "$f_crypto_amount/100*($g_percentage_diff_result+100)"
+        f_crypto_amount_long=$g_calc_result
+        g_calc "$f_crypto_amount/100*($g_percentage_diff_result*-1)"
+        f_crypto_amount_short=$g_calc_result
+        # part short-sell
+        # part long-sell
+        ## Debug
+        #echo PART: process_sell process_sell "$f_symbol" "$f_crypto_amount_long" "$f_fiat_amount_tax_currency_long" "$f_date" "$f_year"
+        process_sell "$f_symbol" "$f_crypto_amount_long" "$f_fiat_amount_tax_currency_long" "$f_date" "$f_year"
+
+        ## Debug
+        #echo PART: process_buy "$f_symbol" "$f_crypto_amount_long" "$f_fiat_amount_tax_currency_long" "$f_date" "$f_year"
+        process_buy "$f_symbol" "$f_crypto_amount_long" "$f_fiat_amount_tax_currency_long" "$f_date" "$f_year" 
+      fi
+    # Process sell actions
+    elif [[ $f_action == "sell" || $f_action == "leverage-sell" ||  $f_action == "liquidation" ]]
+    then
+      # check for long or short or log+short
+      if g_num_is_higher_equal "$f_holdings_amount" "$f_crypto_amount"
+      then
+        # long
+        ## Debug
+        #echo process_sell "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date" "$f_year"
+        process_sell "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date" "$f_year"
+      elif g_num_is_higher "$f_holdings_amount" 0
+      then
+        # long+short (partial)
+        # calc long/short parts
+        g_percentage-diff "$f_crypto_amount" "$f_holdings_amount"
+        g_calc "$f_fiat_amount_tax_currency/100*($g_percentage_diff_result+100)"
+        f_fiat_amount_tax_currency_long=$g_calc_result
+        g_calc "$f_fiat_amount_tax_currency/100*(($g_percentage_diff_result*-1))"
+        f_fiat_amount_tax_currency_short=$g_calc_result
+        g_calc "$f_crypto_amount/100*($g_percentage_diff_result+100)"
+        f_crypto_amount_long=$g_calc_result
+        g_calc "$f_crypto_amount/100*($g_percentage_diff_result*-1)"
+        f_crypto_amount_short=$g_calc_result
+        # part long-sell
+        ## Debug
+        #echo PART: process_sell "$f_symbol" "$f_crypto_amount_long" "$f_fiat_amount_tax_currency_long" "$f_date" "$f_year"
+        process_sell "$f_symbol" "$f_crypto_amount_long" "$f_fiat_amount_tax_currency_long" "$f_date" "$f_year"
+        # part short-sell
+        ## Debug
+        #echo PART: process_buy "$f_symbol" "$f_crypto_amount_short" "$f_fiat_amount_tax_currency_short" "$f_date" short
+        process_buy "$f_symbol" "$f_crypto_amount_short" "$f_fiat_amount_tax_currency_short" "$f_date" short
+      elif [[ $f_action == "liquidation" ]]
+      then
+        # short sell/liquidation
+        ## Debug
+        #echo process_sell "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date"  $f_year short
+        process_sell "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date"  $f_year short
+      else
+        # short buy
+        ## Debug
+        #echo process_buy "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date" short
+        process_buy "$f_symbol" "$f_crypto_amount" "$f_fiat_amount_tax_currency" "$f_date" short
+      fi
+    fi
+   
+    ## DEBUG output
+    #get_holdings_amount
+    #echo "f_holdings_amount=$f_holdings_amount"
+    #echo "============================" 
+   
+  done < "$f_csv_file"
+}
+
+function process_buy {
+  local f_symbol="$1" f_amount="$2" f_price="$3" f_date="$4" f_short="$5"
+  local f_tax_type f_trade_tax
+  # Add to holdings
+  # long
+  [ -z "$f_short" ] && f_holdings[$f_symbol]+="$f_amount:$f_price:$f_date "
+  # short
+  if [ -n "$f_short" ]
+  then
+    f_holdings[$f_symbol]+="-$f_amount:$f_price:$f_date "
+    f_action="${f_action}-short"
+    ## Debug
+    #echo ACTION:$f_action
+  elif [[ $f_action == "reward-staking" ]]
+  then
+    f_tax_type="Sonst-Einkünfte-Staking"
+  elif [[ $f_action == "giveaway" ]]
+  then
+    f_tax_type="Sonst-Einkünfte-Giveaway"
+  elif [[ $f_action == "instant_trade_bonus" ]]
+  then
+    f_tax_type="Kapitalertrag-Instant-Trade-Bonus"
+  fi
+
+  if [ -n "$f_tax_type" ]
+  then
+    f_trade_tax=$f_price
+    f_fiat_amount=0
+    f_fiat_amount_tax_currency=0
+  fi
+
+  
+  echo "$f_date,$f_exchange,$f_action,$f_symbol,$f_amount,$f_fiat_currency,-$f_price,,,,,,$f_tax_type,$f_trade_tax,-$f_price,,,,,,,,," >>ALL_TRANSACTIONS_OVERVIEW.csv.tmp
+}
+
+function process_sell {
+  local f_symbol="$1" f_sell_amount="$2" f_sell_price="$3" f_sell_date="$4" f_year="$5" f_short="$6"
+  f_remaining_sell=$f_sell_amount
+  local f_profit=0 f_loss=0 f_profit_tax=0 f_loss_tax=0 f_trade_tax=0
+  local f_pnl
+
+  # define tax type
+  local f_tax_type="Kapitalertrag-Derivat"
+  local f_trade_result
+  [[ $f_action == "sell" ]] && f_tax_type="Veräußerungsgeschäft"
+  local f_buy_amount f_buy_price f_buy_date
+
+  # Process each holding using FIFO
+  while [[ $f_remaining_sell > 0 && -n "${f_holdings[$f_symbol]}" ]]
+  do
+
+    IFS=':' read -r f_buy_amount f_buy_price f_buy_date < <(echo "${f_holdings[$f_symbol]%% *}")
+
+    # Calculate amount to sell from this holding
+    f_sell_from_holding=$f_buy_amount
+    [ -n "$f_short" ] && f_remaining_sell=-${f_remaining_sell#-}
+    [ -z "$f_short" ] && g_num_is_lower $f_remaining_sell $f_buy_amount && f_sell_from_holding=$f_remaining_sell
+    [ -n "$f_short" ] && g_num_is_higher $f_remaining_sell $f_buy_amount && f_sell_from_holding=$f_remaining_sell
+
+    # calculate sell percentage of buy trade    
+    ## Debug
+    #echo "f_sell_from_holding=$f_sell_from_holding"
+    [ -z "$f_short" ] && g_percentage-diff $f_buy_amount $f_sell_from_holding
+    [ -n "$f_short" ] && g_percentage-diff $f_buy_amount $f_sell_from_holding
+    g_calc "100+$g_percentage_diff_result"
+    f_percentage_of_buy=${g_calc_result#-}
+    
+    ## Debug
+    #echo "f_percentage_of_buy=$f_percentage_of_buy"
+
+    # Calculate profit/loss (pnl)
+    ## Debug
+    #echo "f_sell_price=$f_sell_price"
+    #echo "f_buy_price=$f_buy_price"
+    # if not first iteration (f_pnl is already set from previous iteration) and partial sell
+    if [ -n "$f_pnl" ]
+    then
+      # on multiple iteration partial sell 
+      g_calc "$f_pnl - ($f_buy_price/100*$f_percentage_of_buy)"
+    else
+      # on first interation partial sell
+      g_calc "$f_sell_price - ($f_buy_price/100*$f_percentage_of_buy)"
+    fi
+    local f_pnl=$g_calc_result
+    ## Debug
+    #echo "f_pnl=$f_pnl"    
+
+    # Check if trade is tax-free (held for more than a year)
+    local f_is_taxable=true
+    if [ "$f_tax_type" == "Veräußerungsgeschäft" ]
+    then
+      local f_days_held=$(( ($(date -d "$f_sell_date" +%s) - $(date -d "$f_buy_date" +%s)) / 86400 ))
+      [[ $f_days_held -gt 365 && ${f_tax_type} == "Veräußerungsgeschäft" ]] && f_is_taxable=false
+    fi
+
+    # Update remaining sell amount and holdings
+    g_calc "$f_remaining_sell - $f_sell_from_holding"
+    f_remaining_sell=$g_calc_result
+    ## Debug
+    #echo "f_remaining_sell=$f_remaining_sell"
+    #echo "HOLDINGS1: ${f_holdings[$f_symbol]}"
+    f_holdings[$f_symbol]="${f_holdings[$f_symbol]#* }"
+
+    # If there's remaining amount in the holding, add it back
+    [ -z "$f_short" ] && g_calc "$f_buy_amount - $f_sell_from_holding"
+    [ -n "$f_short" ] && g_calc "$f_buy_amount + $f_sell_from_holding"
+    g_calc "$f_buy_amount - $f_sell_from_holding"
+    local f_remaining_buy_amount=$g_calc_result
+    ## Debug
+    #echo "f_remaining_buy_amount=$g_calc_result"
+    #if g_num_is_higher $f_remaining_buy_amount 0
+    if [ "$f_remaining_buy_amount" != "0" ]
+    then
+      g_calc "$f_buy_price/100*(100-$f_percentage_of_buy)"
+      f_remaining_buy_price=$g_calc_result
+      f_holdings[$f_symbol]="$f_remaining_buy_amount:$f_remaining_buy_price:$f_buy_date ${f_holdings[$f_symbol]}"
+    fi
+    ## Debug
+    #echo "HOLDINGS2: ${f_holdings[$f_symbol]}"
+  done
+
+  # Update profit/loss
+  [ -n "$f_short" ] && g_calc "$f_pnl * -1" && f_pnl=$g_calc_result
+  if g_num_is_higher $f_pnl 0
+  then
+    g_calc "$f_profit + $f_pnl"
+    f_profit=$g_calc_result
+  else
+    g_calc "$f_loss - $f_pnl"
+    f_loss=$g_calc_result
+  fi
+
+  # calculate result of trade
+  g_calc "$f_profit - $f_loss"
+  f_trade_result=$g_calc_result
+
+  # calculate taxable part of trade
+  if [[ $f_is_taxable == true ]]
+  then
+    g_calc "$f_trade_tax + $f_pnl"
+    f_trade_tax=$g_calc_result
+  fi
+
+  ## DEBUG output
+  #get_holdings_amount
+  #echo "f_holdings_amount=$f_holdings_amount"
+  #echo "Result: $f_trade_result ; taxable=$f_is_taxable ; REMAINING: $f_holdings_amount"
+
+  # write to csv
+  if [ -n "$f_short" ]
+  then
+    f_action="${f_action}-short"
+    ## Debug
+    #echo ACTION:$f_action
+  fi 
+  [ "$f_trade_tax" == "0" ] && [ "$f_tax_type" == "Veräußerungsgeschäft" ] && f_tax_type="Veräußerungsgeschäft Spekulationsfrist > 1 Jahr"
+  echo "$f_date,$f_exchange,$f_action,$f_symbol,-$f_sell_amount,$f_fiat_currency,$f_sell_price,,,,,,$f_tax_type,$f_trade_tax,$f_sell_price,,$f_trade_result,,,,,,," >>ALL_TRANSACTIONS_OVERVIEW.csv.tmp
+
+  [ -z "$f_trade_result" ] && g_echo_error "No trade result!!! Someting wrong $f_date,$f_symbol,$f_action $f_short"
+
+}
+
+function get_holdings_amount {
+  local block first_value
+  f_holdings_amount=0
+
+  # Durch jeden Block iterieren
+  IFS=" "
+  for block in ${f_holdings[$f_symbol]}
+  do
+    IFS=$origIFS
+    # Den ersten Wert vor dem Doppelpunkt extrahieren
+    first_value=${block%%:*}
+
+    # Zum Gesamtwert addieren
+    g_calc "$f_holdings_amount + $first_value"
+    f_holdings_amount=$g_calc_result
+  done
+  IFS=$origIFS
+}
+
+function process_fundingfee {
+  local f_symbol="$1" f_amount="$2" f_fiat_amount_tax_currency="$3" f_date="$4" f_year="$5"
+  
+  ## Debug
+  #echo "adding fundingfee: $f_fiat_amount_tax_currency"
+
+  ## add fundingfee
+  [[ $f_fiat_amount_tax_currency == -* ]] && f_tax="${f_fiat_amount_tax_currency#-}"
+  [[ $f_fiat_amount_tax_currency == -* ]] || f_tax="-${f_fiat_amount_tax_currency}"
+  
+  echo "$f_date,$f_exchange,$f_action,$f_symbol,$f_amount,,,,,,,,Kapitalertrag-Derivat,$f_tax,$f_tax,,$f_tax,,,,,,," >>ALL_TRANSACTIONS_OVERVIEW.csv.tmp
+}
+
+#function transaction_csv_validity_ckecks {
+#  local f_buy f_sell f_liquidation f_liquidation_short
+#  local f_complete_result=0
+#  declare -A transaction_csv_validity_ckeck_buy_sell_diff
+#
+#  f_symbols=$(cut -d, -f3 $f_csv_file | sort -u)
+#  local f_buy_amount f_sell_amount f_tax_type 
+#
+#  # go through symbols and male some pre-checks
+#  for f_symbol in $f_symbols
+#  do
+#
+#    ## check asset amount
+#    g_echo_note "Initial checks for $f_symbol"
+#    # add all buys and sells of a symbols amount
+#    f_buy=$(\
+#      egrep "buy,${f_symbol},|,reward-staking,${f_symbol}|,giveaway,${f_symbol},instant_trade_bonus,${f_symbol}" "$f_csv_file" | \
+#      cut -d, -f4 | \
+#      awk '{ SUM += $1} END { printf("%.12f\n", SUM) }' \
+#    )
+#    f_sell=$(\
+#      egrep "sell,${f_symbol}," "$f_csv_file" | \
+#      cut -d, -f4 | \
+#      awk '{ SUM += $1} END { printf("%.12f\n", SUM) }' \
+#    )
+#    f_liquidation=$(\
+#      egrep "liquidation,${f_symbol}," "$f_csv_file" | \
+#      grep -v ",short" | \
+#      cut -d, -f4 | \
+#      awk '{ SUM += $1} END { printf("%.12f\n", SUM) }' \
+#    )
+#    f_liquidation_short=$(\
+#      egrep "liquidation,${f_symbol},.+,short" "$f_csv_file" | \
+#      cut -d, -f4 | \
+#      awk '{ SUM += $1} END { printf("%.12f\n", SUM) }' \
+#    )
+#
+#    # add liquidations to sell
+#    # long
+#    g_calc "$f_sell + $f_liquidation - $f_liquidation_short"
+#    f_sell=$g_calc_result
+#
+#    # buy should be same as sell sum to be fine - if not:
+#    g_calc "$f_buy == $f_sell"
+#    if ! [[ $g_calc_result == 1 ]]
+#    then
+#      g_echo_note "buy ($f_buy) and sell ($f_sell) amount sums are different for ${f_symbol}. Open Positions!?"
+#      g_calc "$f_sell - ($f_buy)" 
+#      transaction_csv_validity_ckecks[$f_symbol]=$g_calc_result
+#    else
+#      transaction_csv_validity_ckeck_buy_sell_diff[$f_symbol]=0
+#    fi
+#
+#  done
+#}
+
+function print_results {
+
+  local f_csv=ALL_TRANSACTIONS_OVERVIEW.csv
+  local f_exchange_symbol f_exchange_symbol_year_tax f_amount f_result
+  #transaction_csv_validity_ckecks
+ 
+  echo "" 
+  echo "Open Positions:"
+  echo "==============="
+
+  local f_exchanges_symbols=$(cut -d, -f 2,4 "$f_csv" | sort -u)
+  for f_exchange_symbol in $f_exchanges_symbols
+  do
+    f_exchange=${f_exchange_symbol%%,*}
+    f_symbol=${f_exchange_symbol#*,}
+    f_amount=$(\
+      egrep ",${f_exchange},.+,$f_symbol" "$f_csv" | \
+      cut -d, -f5 | \
+      awk '{ SUM += $1} END { printf("%.12f\n", SUM) }' \
+    )
+   
+    f_result=$(\
+      egrep ",${f_exchange},.*sell,$f_symbol|,${f_exchange},.*buy,$f_symbol" "$f_csv" | \
+      #egrep "${f_exchange},.+,$f_symbol" "$f_csv" \
+      cut -d, -f17 | \
+      awk '{ SUM += $1} END { printf("%.2f\n", SUM) }' \
+    )
+
+    g_calc "$f_amount == 0"
+    if ! [[ $g_calc_result == 1 ]]
+    then
+      echo "$f_exchange/$f_symbol: $f_amount"
+    fi
+    
+  done
+ 
+  echo ""
+  echo "Profit and Loss (Tax):"
+  echo "======================"
+
+  declare -A f_taxes f_pnls
+  local f_total_tax
+  local f_exchanges_symbols_years_tax=$(sed 's/-/,/' "$f_csv" | cut -d, -f 1,3,5,14 | sort -u)
+  for f_exchange_symbol_year_tax in $f_exchanges_symbols_years_tax
+  do
+    IFS=',' read -r f_year f_exchange f_symbol f_tax_type < <(echo "$f_exchange_symbol_year_tax")
+    [ -z "$f_tax_type" ] && continue
+    f_tax=$(\
+      egrep "^$f_year-.+,${f_exchange},.+,${f_symbol},.+,$f_tax_type" "$f_csv" | \
+      cut -d, -f14 | \
+      awk '{ SUM += $1} END { printf("%.2f\n", SUM) }' \
+    )
+    f_pnl=$(\
+      egrep "^$f_year-.+,${f_exchange},.+,${f_symbol},.+,$f_tax_type" "$f_csv" | \
+      cut -d, -f17 | \
+      awk '{ SUM += $1} END { printf("%.2f\n", SUM) }' \
+    )
+    echo "$f_year/$f_exchange/$f_symbol/$f_tax_type: $f_tax $TRANSFER_CURRENCY"
+    [ -z "${f_taxes[${f_year}_${f_exchange}_${f_tax_type}]}" ] && f_taxes[${f_year}_${f_exchange}_${f_tax_type}]=0
+    [ -z "${f_pnls[${f_year}_${f_exchange}]}" ] && f_pnls[${f_year}_${f_exchange}]=0
+    g_calc "${f_taxes[${f_year}_${f_exchange}_${f_tax_type}]} + ($f_tax)"
+    f_taxes[${f_year}_${f_exchange}_${f_tax_type}]=$g_calc_result
+    g_calc "${f_pnls[${f_year}_${f_exchange}]} + ($f_pnl)"
+    f_pnls[${f_year}_${f_exchange}]=$g_calc_result
+
+  done
+
+  echo ""
+  echo "Profit and Loss (Tax per exchange):"
+  echo "==================================="
+
+  for f_tax_year in "${!f_taxes[@]}"
+  do
+    echo "$f_tax_year: ${f_taxes[$f_tax_year]} $TRANSFER_CURRENCY"
+  done | sort
+
+  echo ""
+  echo "Profit and Loss:"
+  echo "================"
+
+  for f_pnl_year in "${!f_pnls[@]}"
+  do
+    echo "$f_pnl_year: ${f_pnls[$f_pnl_year]} $TRANSFER_CURRENCY"
+  done | sort
+
+}
+
+
--- a/dabo/functions/currency_converter.sh
+++ b/dabo/functions/currency_converter.sh
@@ -26,14 +26,20 @@ function currency_converter {
  local f_currency=$2
  local f_currency_target=$3
  local f_currency_date=$4
-  
+  local f_return
+
  unset f_currency_converter_result

+  # check for cached result
+  local f_args=$@
+  [ -f CACHE_CURRENCY_CONVERTER ] && f_currency_converter_result=$(egrep "^${f_args}=" CACHE_CURRENCY_CONVERTER | cut -d= -f2)
+  [[ -n $f_currency_converter_result ]] && g_num_valid_number "$f_currency_converter_result" && return 0
+
  local f_line f_rate f_histfile f_date_array f_stablecoin f_reverse f_file f_link_file f_timeframe

  if [[ $f_currency_target =~ ^20.*-.*: ]]
  then
-    g_echo_warn "${FUNCNAME}: Invalid target $f_currency_target"
+    g_echo_warn "${FUNCNAME} $@: Invalid target"
    g_traceback
    return 1
  fi
@@ -51,7 +57,13 @@ function currency_converter {
  f_currency_date_day=$(date -d "${f_currency_date}" "+%Y-%m-%d")

  # month failback
-  f_currency_date_month=$(date -d "${f_currency_date}" "+%Y-%m")
+  if [ $(date -d "${f_currency_date}" "+%d") = "01" ]
+  then
+    # on first day in month use month before because no date from current month
+    f_currency_date_month=$(date -d "${f_currency_date} yesterday" "+%Y-%m")
+  else  
+    f_currency_date_month=$(date -d "${f_currency_date}" "+%Y-%m")
+  fi

  # path to history files for the converting rate
  [ -d asset-histories ] || mkdir asset-histories
@@ -79,6 +91,29 @@ function currency_converter {
      f_currency=USD
    fi
  done
+  
+  # map EUR-Stablecoins to EUR
+  local f_stablecoins="EURC"
+  for f_stablecoin in $f_stablecoins
+  do
+    # Link EUR Stablecoin files to EUR
+    cd "$f_asset_histories"
+    find . -maxdepth 1 -mindepth 1 -name "*${f_stablecoin}.history.*.csv" | while read f_file
+    do
+      f_link_file=${f_file/${f_stablecoin}/EUR}
+      ln -sf "$f_file" "$f_link_file"
+    done
+    cd - >/dev/null
+    # use USD
+    if [[ $f_currency_target = $f_stablecoin ]]
+    then
+      f_currency_target=EUR
+    fi
+    if [[ $f_currency = $f_stablecoin ]]
+    then
+      f_currency=EUR
+    fi
+  done

  # if there is no currency change (USD USD or USDT USD)
  if [[ $f_currency == $f_currency_target ]]
@@ -106,7 +141,7 @@ function currency_converter {
    for f_timeframe in 1d 1w
    do
      [ "${f_currency}" = "USD" ] && get_marketdata_coinmarketcap "${f_currency_target}-${f_currency}" "${f_currency_target}${f_currency}" $f_timeframe
-      [ "${f_currency_target}" = "USD" ] && get_marketdata_coinmarketcap "${f_currency}-${f_currency_target}" "${f_currency}${f_currency_target}"
+      [ "${f_currency_target}" = "USD" ] && get_marketdata_coinmarketcap "${f_currency}-${f_currency_target}" "${f_currency}${f_currency_target}" $f_timeframe
    done
    f_line=$(egrep "^$f_currency_date_minute" "$f_histfile"*m.csv 2>/dev/null | sort | tail -n1)
    [ -z "$f_line" ] && f_line=$(egrep "^$f_currency_date_hour" "$f_histfile"*m.csv 2>/dev/null | sort | tail -n1)
@@ -123,24 +158,34 @@ function currency_converter {
    [[ $f_histfile =~ ${f_currency}${f_currency_target} ]] && f_reverse=true
    [ $f_currency_target = "USD" ] && f_reverse=true
    [ $f_currency = "USD" ] && f_reverse=false
-    [ $f_currency_target = "EUR" ] && [ $f_currency = "USD" ] &&  f_reverse=true
+    [ $f_currency_target = "EUR" ] && [ $f_currency = "USD" ] &&  f_reverse=false
    [[ $f_line =~ ^$f_currency_date_hour ]] && break
  fi

-  # end if no rate found
+  # if no rate found
  if [ -z "$f_rate" ]
  then
-    # try workaround over USD if EUR
-    if [[ ${f_currency_target} = EUR ]] && [[ $f_currency != USD ]]
+    # if EUR source or traget try way over USD as workaround
+    if [[ ${f_currency_target} = EUR ]] && [[ $f_currency != USD ]] &&  [[ $f_currency != EUR ]] 
    then
-      #g_echo_note "trying way over USD (workaround)"
-      if currency_converter $f_currency_amount $f_currency USD $f_currency_date
+      g_echo_note "trying way over USD (workaround) Target EUR"
+      if currency_converter $f_currency_amount $f_currency USD "$f_currency_date"
      then
-        f_currency_amount=$f_currency_converter_result
-        currency_converter $f_currency_amount USD EUR $f_currency_date
-        return $?
+        currency_converter $f_currency_converter_result USD EUR "$f_currency_date" && f_return=$?
+        [[ $f_return == 0 ]] && echo "$@=$f_currency_converter_result" >>CACHE_CURRENCY_CONVERTER
+        return $f_return
+      fi
+    elif [[ ${f_currency_target} != USD ]] && [[ ${f_currency_target} != EUR ]] && [[ $f_currency = EUR ]]
+    then
+      g_echo_note "trying way over USD (workaround) Source EUR"
+      if currency_converter $f_currency_amount EUR USD "$f_currency_date"
+      then
+        currency_converter $f_currency_converter_result USD ${f_currency_target} "$f_currency_date" && f_return=$?
+        [[ $f_return == 0 ]] && echo "$@=$f_currency_converter_result" >>CACHE_CURRENCY_CONVERTER
+        return $f_return
      fi
    fi
+    # end if no rate found
    g_echo_error "didn't find rate for ${f_currency}-${f_currency_target} - '${FUNCNAME} $@'"
    return 1
  fi
@@ -150,8 +195,6 @@ function currency_converter {
  [[ $f_reverse = false ]] && g_calc "1/${f_rate}*${f_currency_amount}"
  f_currency_converter_result=$g_calc_result

+  echo "$@=$f_currency_converter_result" >>CACHE_CURRENCY_CONVERTER
+
 }
-
-
-
-
--- a/dabo/functions/get_indicators.sh
+++ b/dabo/functions/get_indicators.sh
@@ -26,7 +26,8 @@ function get_indicators_all {
  
  local f_histfile f_symbol

-  find asset-histories -maxdepth 1 -name "ECONOMY-*.history.[0-5][5dhwm]*.csv" | sort | while read f_histfile
+  # ECONOMY and MARKETDATA
+  find asset-histories -maxdepth 1 -name "ECONOMY-*.history.[0-5][5dhwm]*.csv" -o -name "MARKETDATA_*.history.[0-5][5dhwm]*.csv" | sort | while read f_histfile
  do
    if [ -s "${f_histfile}.fetching" ] || [ -s "${f_histfile}.indicators-calculating" ]
    then
@@ -35,8 +36,8 @@ function get_indicators_all {
    fi

    # do the job
-    printf '%(%Y-%m-%d %H:%M:%S)T' >"${f_histfile}.indicators-calculating"
-    get_indicators "${f_histfile}" ${f_last_intervals} && printf '%(%Y-%m-%d %H:%M:%S)T' >>"$f_histfile.indicators-calculated"
+    printf "$0 %(%Y-%m-%d %H:%M:%S)T" >"${f_histfile}.indicators-calculating"
+    get_indicators "${f_histfile}" ${f_last_intervals} && printf "$0 %(%Y-%m-%d %H:%M:%S)T\n" >>"$f_histfile.indicators-calculated"
    # add missing intervals for example from weekends from non-24h-assets like economic data - interval from filename
    #f_add_missing_ohlcv_intervals "${f_histfile}"
    rm -f "${f_histfile}.indicators-calculating"
@@ -62,14 +63,14 @@ function get_indicators_all {
        fi
      
        # do the job
-        printf '%(%Y-%m-%d %H:%M:%S)T' >"${f_histfile}.indicators-calculating"     
-        get_indicators "${f_histfile}" ${f_last_intervals} && printf '%(%Y-%m-%d %H:%M:%S)T' >>"$f_histfile.indicators-calculated"
+        printf "$0 %(%Y-%m-%d %H:%M:%S)T" >"${f_histfile}.indicators-calculating"
+        get_indicators "${f_histfile}" ${f_last_intervals} && printf "$0 %(%Y-%m-%d %H:%M:%S)T\n" >>"$f_histfile.indicators-calculated"
        rm -f "${f_histfile}.indicators-calculating"

      fi
    done
  done
-  shopt +s nullglob
+  shopt -u nullglob

 }

@@ -82,17 +83,27 @@ function get_indicators {
  local f_fill_missing_ohlcv_intervals=$3
  local f_line 

+  # check if the job is already done
+  if [ $(wc -l <"${f_histfile}") -gt 801 ] 
+  then
+    if ! tail +801 "${f_histfile}" | egrep -vq "^....-..-..*,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.\-]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.]+,[0-9\.\-]+,[0-9\.\-]+,[0-9\.\-]+,[a-z]*,[0-9\.\-]+,[0-9\.\-]+"
+    then
+      g_echo_note "${f_histfile} seems to be indicator-complete"
+      return 0
+    fi
+  fi
+  
  # history
  local f_columns="date,open,high,low,close,volume,change,ath,ema12,ema26,ema50,ema100,ema200,ema400,ema800,rsi5,rsi14,rsi21,macd,macd_ema9_signal,macd_histogram,macd_histogram_signal,macd_histogram_max,macd_histogram_strength"
  local f_emas="12 26 50 100 200 400 800"
  local f_rsis="5 14 21"
-  local f_ema f_change f_changed f_line f_valid_data f_ath
+  local f_ema f_change f_changed f_line f_valid_data f_ath f_last_year
  local f_columns_space="${f_columns//,/ }"
  g_read_csv "${f_histfile}" "${f_last_intervals}" "$f_columns"
  for ((i=0; i<=${#g_csv_array[@]}-1; i++))
  do

-    if [ -z "${v_csv_array_associative[date_${i}]}" ] 
+    if [ -z "${v_csv_array_associative[date_${i}]}" ] || [ -z "${v_csv_array_associative[open_${i}]}" ]
    then
      g_echo_note "No data $f_histfile:${v_csv_array_associative[date_${i}]}"
      return 0
@@ -105,16 +116,42 @@ function get_indicators {
    ### check for unfilled fields
    f_change=""

+    # check olhc data
+    if [ -z "${v_csv_array_associative[high_${i}]}" ] && [ -z "${v_csv_array_associative[low_${i}]}" ] && [ -z "${v_csv_array_associative[close_${i}]}" ]
+    then
+      g_echo_note "fixing OHLC Data"
+      # open is close
+      v_csv_array_associative[close_${i}]=${v_csv_array_associative[open_${i}]}
+      # open is previous close
+      [ -n "${v_csv_array_associative[close_${p}]}" ] && v_csv_array_associative[open_${i}]="${v_csv_array_associative[close_${p}]}"
+      # calc high/low from open/close 
+      if g_num_is_higher_equal ${v_csv_array_associative[open_${i}]}  ${v_csv_array_associative[close_${i}]}
+      then
+        v_csv_array_associative[high_${i}]=${v_csv_array_associative[open_${i}]}
+        v_csv_array_associative[low_${i}]=${v_csv_array_associative[close_${i}]}
+      else
+        v_csv_array_associative[high_${i}]=${v_csv_array_associative[close_${i}]}
+        v_csv_array_associative[low_${i}]=${v_csv_array_associative[open_${i}]}
+      fi
+      f_change=1
+    fi
+    
    # check for missing percentage change
    if [ -z "${v_csv_array_associative[change_${i}]}" ]
    then
-      #if ! [ $p -lt 0 ] 
-      #then
-        #echo "g_percentage-diff ${v_csv_array_associative[close_${p}]} ${v_csv_array_associative[close_${i}]}"
-        #g_percentage-diff ${v_csv_array_associative[close_${p}]} ${v_csv_array_associative[close_${i}]} && f_change=1
-        g_percentage-diff ${v_csv_array_associative[open_${i}]} ${v_csv_array_associative[close_${i}]} && f_change=1
-        v_csv_array_associative[change_${i}]=${g_percentage_diff_result}
-      #fi
+      # special for changes watched per year like CPI,...
+      if [[ $f_histfile = "asset-histories/MARKETDATA_US_CONSUMER_PRICE_INDEX_CPI.history.1d.csv" ]] || \
+       [[ $f_histfile = "asset-histories/MARKETDATA_US_UNEMPLOYMENT_RATE.history.1d.csv" ]]
+      then
+        if [ $i -ge 12 ]
+        then
+          f_last_year=$((i-12))
+          g_percentage-diff ${v_csv_array_associative[open_${f_last_year}]} ${v_csv_array_associative[close_${i}]}
+          v_csv_array_associative[year_change_${i}]=${g_percentage_diff_result}
+        fi
+      fi
+      g_percentage-diff ${v_csv_array_associative[open_${i}]} ${v_csv_array_associative[close_${i}]} && f_change=1
+      v_csv_array_associative[change_${i}]=${g_percentage_diff_result}
    fi
    
    # ath (all-time-high) of present data
@@ -135,10 +172,20 @@ function get_indicators {
    # check for missing EMAs
    for f_ema_column in $f_emas
    do
-      # check for enough values/lines to calculate EMA
-      [ $i -ge $f_ema_column ] || continue
+      # check for enough values/lines to calculate EMA if no previous EMA given
+      if [ -z "${v_csv_array_associative[ema${f_ema_column}_${p}]}" ]
+      then
+        if ! [ $i -ge $f_ema_column ] 
+        then
+          #echo "not enough lines $i -ge $f_ema_column"
+          continue
+        fi
+      fi
      # calculate EMA
-      [ -z "${v_csv_array_associative[ema${f_ema_column}_${i}]}" ] && calc_ema ${f_ema_column} close && f_change=1
+      if [ -z "${v_csv_array_associative[ema${f_ema_column}_${i}]}" ] 
+      then
+        calc_ema ${f_ema_column} close && f_change=1
+      fi
    done

    # check for missing RSI
@@ -164,6 +211,11 @@ function get_indicators {
      # build line
      for f_column in $f_columns
      do
+        # special for changes watched per year like CPI,...
+        if [[ $f_column = change ]]
+        then
+          [ -n "${v_csv_array_associative[year_change_${i}]}" ] && v_csv_array_associative[change_${i}]=${v_csv_array_associative[year_change_${i}]}
+        fi
        if [ -z "$f_line" ]
        then
          f_line="${v_csv_array_associative[${f_column}_${i}]}"
@@ -178,6 +230,8 @@ function get_indicators {
    fi

  done
-
+  
+  # cleanup large arrays
+  unset v vr v_csv_array_associative v_csv_array_associative_reverse 
 }

--- a/dabo/functions/get_levels.sh
+++ b/dabo/functions/get_levels.sh
@@ -70,6 +70,7 @@ function get_levels {
 
  # reset old levels var
  unset f_levels
+  unset f_zones

  local f_levelsfile=$1
  if ! [ -s "$f_levelsfile" ]
--- a/dabo/functions/get_marketdata.sh
+++ b/dabo/functions/get_marketdata.sh
@@ -17,95 +17,146 @@
 # You should have received a copy of the GNU General Public License
 # along with dabo. If not, see <http://www.gnu.org/licenses/>.

+function get_marketdata_all {
+  local f_interval=$1
+  
+  # daily garketdata jobs
+  if [[ $f_interval = 1d ]]
+  then
+    # FEAR_AND_GREED_ALTERNATIVEME
+    get_marketdata FEAR_AND_GREED_ALTERNATIVEME 'https://api.alternative.me/fng/?limit=0&format=json' '.data[] | (.timestamp | tonumber | strftime("%Y-%m-%d")) + "," + .value + ",,,,0"' "" 1d
+
+    # FEAR AND GREED COINMARKETCAP
+    get_marketdata FEAR_AND_GREED_COINMARKETCAP "https://api.coinmarketcap.com/data-api/v3/fear-greed/chart?start=1&end=$(date +%s)" '.data.dataList[] | (.timestamp | tonumber | strftime("%Y-%m-%d")) + "," + (.score|tostring) + ",,,,0"'
+
+    # FEAR_AND_GREED_CNN
+    get_marketdata FEAR_AND_GREED_CNN 'https://production.dataviz.cnn.io/index/fearandgreed/graphdata' '.fear_and_greed_historical.data[] | (.x/1000 | strftime("%Y-%m-%d")) + "," + (.y|tostring) + ",,,,0"' "" 1d
+
+    # Altcoin-Saison-Index COINMARKETCAP Top 100 Altcoins
+    get_marketdata ALTCOIN_SEASON_INDEX_COINMARKETCAP "https://api.coinmarketcap.com/data-api/v3/altcoin-season/chart?start=1&end=$(date +%s)" '.data.points[:-1][] | (.timestamp | tonumber | strftime("%Y-%m-%d")) + "," + (.altcoinIndex|tostring) + ",,,,0"' "" 1d
+
+    # monthly US consumer price index CPI data
+    get_marketdata US_CONSUMER_PRICE_INDEX_CPI "https://api.bls.gov/publicAPI/v2/timeseries/data/CUUR0000SA0?startyear=$(date -d 'now -8 years' '+%Y')&endyear=$(date '+%Y')" '.Results.series[0].data[] | .year + "-" + (.period | gsub("M"; ""))  + "-01," + .value + ",,,,0"' "" 1d
+
+    # monthly US unemployment rate
+    get_marketdata US_UNEMPLOYMENT_RATE "https://api.bls.gov/publicAPI/v2/timeseries/data/LNU03000000?startyear=$(date -d 'now -8 years' '+%Y')&endyear=$(date '+%Y')" '.Results.series[0].data[] | .year + "-" + (.period | gsub("M"; ""))  + "-01," + .value + ",,,,0"' "" 1d
+  
+    # US FED funds rate
+    get_marketdata US_FED_FUNDS_RATE 'https://fred.stlouisfed.org/graph/fredgraph.csv?id=DFF' "" "" 1d    
+
+  fi
+
+   # Binance Long Short Ration Account / Taker and Open Interest per symbol
+   get_symbols_ticker
+   local f_symbol f_asset f_time
+   for f_symbol in BTC/$CURRENCY "${f_symbols_array_trade[@]}"
+   do
+     f_asset=${f_symbol//:$CURRENCY/}
+     f_asset=${f_asset//\//}
+
+     # week not available
+     [[ $f_interval = 1w ]] && continue
+     
+     f_time='%Y-%m-%d %H:%M:00'
+     [[ $f_interval = 1d ]] && f_time='%Y-%m-%d'
+   
+     # BINANCE_LONG_SHORT_RATIO_ACCOUNT per symbol
+     get_marketdata BINANCE_LONG_SHORT_RATIO_ACCOUNT_$f_asset "https://fapi.binance.com/futures/data/globalLongShortAccountRatio?symbol=${f_asset}&limit=500&period=${f_interval}" ".[] | (.timestamp/1000 | strftime(\"${f_time}\")) + \",\" + .longShortRatio + \",,,,0\"" "" ${f_interval}
+    
+     # BINANCE_LONG_SHORT_RATIO_Taker per symbol
+     get_marketdata BINANCE_LONG_SHORT_RATIO_TAKER_$f_asset "https://fapi.binance.com/futures/data/takerlongshortRatio?symbol=${f_asset}&limit=500&period=${f_interval}" ".[] | (.timestamp/1000 | strftime(\"${f_time}\")) + \",\" + .buySellRatio + \",,,,0\"" "" ${f_interval}
+
+     # BINANCE_OPEN_INTEREST per symbol
+     get_marketdata BINANCE_OPEN_INTEREST_$f_asset "https://fapi.binance.com/futures/data/openInterestHist?symbol=${f_asset}&limit=500&period=${f_interval}" ".[] | (.timestamp/1000 | strftime(\"${f_time}\")) + \",\" + .sumOpenInterest + \",,,,0\"" "" ${f_interval}
+
+   done
+}
+

 function get_marketdata {
- 
+
  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
 
+  local f_name=$1
+  local f_wget=$2
+  local f_jq=$3
+  local f_other=$4
+  local f_timeframe=$5
+  [ -z "$f_timeframe" ] && f_timeframe=1d
+  local f_histfile="asset-histories/MARKETDATA_${f_name}.history.${f_timeframe}.csv"
+  local f_dataline f_failed

-### CPI Data: curl hapi.bls.gov/publicAPI/v2/timeseries/data/cpi | jq -a
-### release dates: https://www.bls.gov/schedule/news_release/bls.ics
-
-  get_marketdata_from_url https://www.investing.com/economic-calendar/unemployment-rate-300/ US-UNEMPLOYMENT-INDEX
-  get_marketdata_from_url https://www.investing.com/economic-calendar/cpi-733 US-CONSUMER-PRICE-INDEX
-  get_marketdata_from_url https://www.investing.com/indices/fed-funds-composite-interest-rate-opinion US-FED-FEDERAL-FUNDS-RATE-INVERTED-INDEX
-  get_marketdata_from_url '"https://fapi.binance.com/futures/data/globalLongShortAccountRatio?symbol=BTCUSDT&period=5m" | jq  -r .[].longShortRatio | tail -n1' BINANCE-BTC-GlobalLongShortAccountRatio
-  get_marketdata_from_url '"https://fapi.binance.com/futures/data/takerlongshortRatio?symbol=BTCUSDT&period=5m" | jq -r .[].buySellRatio | tail -n1' BINANCE-BTC-TakerLongShortRatio
-  get_marketdata_from_url '"https://fapi.binance.com/futures/data/openInterestHist?symbol=BTCUSDT&period=5m" | jq -r .[].sumOpenInterest | tail -n1' BINANCE-BTC-OpenInterest
-  get_marketdata_from_url '"https://api.alternative.me/fng/?limit=1&format=json" | jq -r .data[].value' CRYPTO_FEAR_AND_GREED-INDEX
-  # clear old stuff
-  find asset-histories/*INDEX* -type f -mtime +6 -delete
-}
-
-function get_marketdata_from_url {
-  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
-
-  local f_url="$1"
-  local f_name="$2"
-
-  ## get data for analysis
-  if find asset-histories/${f_name}.history.csv -mmin -${INTERVAL_MIN} 2>/dev/null | grep -q "asset-histories/${f_name}.history.csv"
+  # download
+  g_wget -O "${f_histfile}.wget.tmp" $f_wget 2>"${f_histfile}.err.tmp" || f_failed=wget
+  [ -s "${f_histfile}.wget.tmp" ] || f_failed=wget
+  if [ -n "$f_failed" ]
  then
-    g_echo_note "asset-histories/${f_name}.history.csv already downloaded in the last ${INTERVAL_MIN} minutes"
-    f_get_marketdata_price=$(cat MARKET_DATA_CMD_OUT-${f_name})
-    return 0
+    echo "g_wget -O \"${f_histfile}.wget.tmp\" $f_wget 2>\"${f_histfile}.err\"" >"${f_histfile}.err"
  fi

-  # check source platform for parsing parameters, prepare and run wget command
-  >MARKET_DATA_CMD
-  if echo "${f_url}" | grep -q "boerse.de" 
+  # jd
+  if [ -z "$f_failed" ] && [ -n "$f_jq" ]
  then
-    echo "wget ${g_wget_opts} -q -O - ${f_url} | egrep 'itemprop=\"price\" content=\"[0-9]+\.[0-9]+\"' | sed s'#\"#\n#g' | egrep '^[0-9]+\.[0-9]+'" >MARKET_DATA_CMD
-  elif echo "${f_url}" | egrep -q "investing.com.+economic-calendar" 
-  then
-    echo "wget ${g_wget_opts} -q -O - ${f_url} | egrep 'Actual.+Forecast.+Previous' | cut -d'>' -f7,11 | cut -d'<' -f1,2 | sed 's#,##g' | sed 's/\%//g' | sed 's#</div>#,#' | grep '[0-9]' | cut -d, -f1" >MARKET_DATA_CMD
-  elif echo "${f_url}" | egrep -q "investing.com.+indices" 
-  then
-    echo "wget ${g_wget_opts} -q -O - ${f_url} | sed 's#</div>#\n#g' | grep 'text-5xl.*font-bold.*md:text-' | sed 's#^.*>##; s#,##g' | grep '[0-9]'" >MARKET_DATA_CMD
-  elif echo "${f_url}" | egrep -q '^"https://'
-  then
-    echo "wget -q -O - ${g_wget_opts} ${f_url}" >MARKET_DATA_CMD
-  else 
-  # default to Yahoo Finace Symbols via API
-    echo "wget ${g_wget_opts} -q -O - https://query1.finance.yahoo.com/v8/finance/chart/${f_url} | jq -r '.[].result[].meta.regularMarketPrice'" >MARKET_DATA_CMD
-  fi
-
-  g_runcmd g_retrycmd sh MARKET_DATA_CMD >MARKET_DATA_CMD_OUT-${f_name}.tmp 2>MARKET_DATA_CMD_OUT-${f_name}.tmp.err
-
-  # check output
-  local f_get_marketdata_price_tmp=$(cat MARKET_DATA_CMD_OUT-${f_name}.tmp)
-  if g_num_valid_number ${f_get_marketdata_price_tmp}
-  then
-    if egrep -q "^0\.00" MARKET_DATA_CMD_OUT-${f_name}.tmp
+    if ! jq -r "$f_jq" "${f_histfile}.wget.tmp" >"${f_histfile}.tmp" 2>"${f_histfile}.err.tmp"
    then
-      g_echo_note "Ignoring ${f_name} $(tail -n 10 MARKET_DATA_CMD_OUT-${f_name}.tmp) - maybe out of business day"
+      echo jq -r "$f_jq" "${f_histfile}.wget.tmp" >"${f_histfile}.err"
+      f_failed=jq
    else
-      g_echo_note "${f_name}: ${f_get_marketdata_price_tmp}"
-      mv MARKET_DATA_CMD_OUT-${f_name}.tmp MARKET_DATA_CMD_OUT-${f_name}
+      mv "${f_histfile}.tmp" "${f_histfile}.wget.tmp"
+    fi
+  fi
+ 
+  # other/additional processing
+  if [ -z "$f_failed" ] && [ -n "$f_other" ]
+  then
+    if ! cat "${f_histfile}.wget.tmp" | eval $f_other
+    then
+      echo "cat \"${f_histfile}.wget.tmp\" | $f_other" >"${f_histfile}.err"
+      f_failed=other
    fi
  else
-    g_echo_warn "MARKET_DATA_CMD_OUT-${f_name}.tmp has wrong Syntax. - Not updating ${f_name} Index.
- 
-CMD:
-$(tail -n 10 MARKET_DATA_CMD)
-
-Output: 
-$(tail -n 10 MARKET_DATA_CMD_OUT-${f_name}.tmp | cat -t) 
-
-Error: 
-$(tail -n 10 MARKET_DATA_CMD_OUT-${f_name}.tmp.err | cat -t)"
+    mv "${f_histfile}.wget.tmp" "${f_histfile}.tmp"
  fi
-  if ! [ -e "MARKET_DATA_CMD_OUT-${f_name}" ]
+
+
+  # cleanup
+  rm -f "${f_histfile}.wget.tmp" "${f_histfile}.err.tmp"
+
+  # error if no csvfile available
+  if [ -n "$f_failed" ] || ! [ -s "${f_histfile}.tmp" ]
  then
-    local f_old_value=$(tail -n 1 asset-histories/${f_name}.history.csv | cut -d, -f2)
-    if echo ${f_old_value} | egrep -q "^[0-9]*\.[0-9]+"
+    cat "${f_histfile}.err.tmp" >>"${f_histfile}.err"
+    cat "${f_histfile}.wget.tmp" >>"${f_histfile}.err"
+    cat "${f_histfile}.err" 1>&2
+    mkdir -p FAILED_MARKETDATA
+    mv "${f_histfile}.err" "FAILED_MARKETDATA/MARKETDATA-${f_name}" 2>/dev/null
+    return 1
+  fi
+
+  # on first download  
+  if ! [ -s "${f_histfile}" ]
+  then
+    grep ^[2-9] "${f_histfile}.tmp" | sort -k1,1 -t, -u >"${f_histfile}"
+  else
+    # merge data
+    egrep -h ^[0-9][0-9][0-9][0-9]-[0-9][0-9] "${f_histfile}" "${f_histfile}.tmp" | sort -k1,1 -t, -u >"${g_tmp}/${FUNCNAME}.tmp"
+
+    # if there is new dataline add it
+    if ! cmp -s "${g_tmp}/${FUNCNAME}.tmp" "${f_histfile}"
    then
-      echo ${f_old_value} >MARKET_DATA_CMD_OUT-${f_name}
-    else
-      echo 0 >MARKET_DATA_CMD_OUT-${f_name}
+      cat "${g_tmp}/${FUNCNAME}.tmp" >"${f_histfile}"
    fi
  fi
-  f_get_marketdata_price=$(cat MARKET_DATA_CMD_OUT-${f_name})
-  echo "${f_timestamp},${f_get_marketdata_price}" >>asset-histories/${f_name}.history.csv
+  rm "${f_histfile}.tmp"
+
+  # calc indicators and if 1d then generate 1w histfile
+  if [[ $f_interval = 1d ]]
+  then
+    get_indicators "${f_histfile}" 51
+    convert_ohlcv_1d_to_1w "${f_histfile}" "${f_histfile/.1d./.1w.}"
+    get_indicators "${f_histfile/.1d./.1w.}" 51
+  else
+    get_indicators "${f_histfile}" 51
+  fi
 }

--- a/dabo/functions/get_marketdata_coinmarketcap.sh
+++ b/dabo/functions/get_marketdata_coinmarketcap.sh
@@ -64,7 +64,7 @@ function get_marketdata_coinmarketcap {

  local f_id
  # get id -> If multiple take the one with the largest marketcap
-  f_id=$(egrep "^${f_item},[1-9]" COINMARKETCAPIDS | sort -n -t, -k4 | tail -n1 | cut -d, -f2)
+  f_id=$(egrep "^${f_item},[1-9]" COINMARKETCAPIDS COINMARKETCAPIDS.tmp 2>/dev/null | sort -n -t, -k4 | tail -n1 | cut -d, -f2)
  [[ $f_item = EURC ]] && f_id=20641
  if [ -z "$f_id" ]
  then
@@ -91,7 +91,6 @@ function get_marketdata_coinmarketcap {
  # cleanup
  rm -f "$f_targetcsvtmp" "${f_targetcsvtmp}".err ${f_targetjsontmp} "${f_targetjsontmp}".err

-
  if [ "$f_timeframe" = "1d" ] || [ "$f_timeframe" = "7d" ]
  then
    # Download data from coinmarketcap
@@ -119,6 +118,9 @@ function get_marketdata_coinmarketcap {
  else
    egrep -h "^[1-9][0-9][0-9][0-9]-[0-1][0-9]-[0-9][0-9].*,[0-9]" "${f_targetcsvtmp}" | sort -k1,2 -t, -u >"$f_targetcsv"
  fi
+  
+  # change exponential notation to normal notation
+  g_num_exponential2normal_file "${f_targetcsv}"

 }

@@ -152,7 +154,7 @@ function get_marketdata_coinmarketcap_ids {
    f_latest_date_seconds=$(date -d "$f_latest_date" +%s)
    if [ $f_latest_date_seconds_now -lt $f_latest_date_seconds ]
    then
-      jq -r '.data | .symbol + "," + (.id|tostring) + "," + .name + "," + (.quotes[].quote|.marketCap|tostring)' "$g_tmp/get_marketdata_coinmarketcap_ids.json"  | head -n 1
+      jq -r '.data | .symbol + "," + (.id|tostring) + "," + .name + "," + (.quotes[].quote|.marketCap|tostring)' "$g_tmp/get_marketdata_coinmarketcap_ids.json"  | grep -vi ",0e-" | head -n 1
    fi
  done | egrep --line-buffered '^.+,[0-9]*,' >"$f_target_loop"
  
--- a/dabo/functions/get_marketdata_yahoo.sh
+++ b/dabo/functions/get_marketdata_yahoo.sh
@@ -32,6 +32,7 @@ function get_marketdata_yahoo {

  [ -z "$f_timeframe" ] && f_timeframe="1d"
  local f_targetcsv="asset-histories/${f_name}.history-yahoo.${f_timeframe}.csv"
+  local f_targetbotcsv="asset-histories/${f_name}.history.${f_timeframe}.csv"
  [ "$f_timeframe" = "1w" ] && f_timeframe="1wk"
  f_histfile_yahoo="$f_targetcsv"

@@ -86,38 +87,48 @@ function get_marketdata_yahoo {
  # cleanup
  rm -f "$f_targetcsvtmp" "${f_targetcsvtmp}".err ${f_targetjsontmp} "${f_targetjsontmp}".err

-  if [ "$f_timeframe" = "1d" ] || [ "$f_timeframe" = "1wk" ] || [ "$f_timeframe" = "1mo" ]
-  then
-    ### DEPRECATED - Yahoo Finance deactivated public API 2024-09 !!! ###
-    g_echo_note "DEPRECATED - Yahoo Finance deactivated public API 2024-09 !!!"
-    return 1
-    ## restrict to finished candles
-    #[ "$f_timeframe" = "1d" ] && f_sec=$(TZ=US/NY date -d 'last day 0:00' '+%s')
-    #[ "$f_timeframe" = "1wk" ] && f_sec=$(TZ=US/NY date -d 'last monday 0:00' '+%s')
-    #[ "$f_timeframe" = "1mo" ] && f_sec=$(TZ=US/NY date -d "$(date -d "@$(date -d "last month" +%s)" +'%Y-%m-01')" +%s)
-    ## Download historical data from yahoo
-    #g_wget -O "${f_targetcsvtmp}" "https://query1.finance.yahoo.com/v7/finance/download/${f_item}?period1=0&period2=${f_sec}&interval=${f_timeframe}&events=history" 2>"${f_targetcsvtmp}".err
-  else
-    # Download data from yahoo
-    g_wget -O "${f_targetjsontmp}" "https://query1.finance.yahoo.com/v7/finance/chart/${f_item}?interval=${f_timeframe}&period2=${f_sec}" 2>"${f_targetjsontmp}".err
-    jq -r '.chart.result[0] as $result | range(0; $result.timestamp | length) | [$result.timestamp[.], $result.indicators.quote[0].open[.], $result.indicators.quote[0].high[.], $result.indicators.quote[0].low[.], $result.indicators.quote[0].close[.], $result.indicators.quote[0].volume[.]] | @csv' "${f_targetjsontmp}" >"${f_targetcsvtmp}.unixtime" 2>"${f_targetjsontmp}".err
+  local f_from
+  [ "$f_timeframe" = "5m" ] && f_from=$(date -d "now -86000 minutes" +%s)
+  [ "$f_timeframe" = "15m" ] && f_from=$(date -d "now -86000 minutes" +%s)
+  [ "$f_timeframe" = "1h" ] && f_from=$(date -d "now -17510 hour" +%s)
+  [ "$f_timeframe" = "1d" ] && f_from=1
+  [ "$f_timeframe" = "1wk" ] && f_from=1
+  [ "$f_timeframe" = "1mo" ] && f_from=1

-    # change unix time to human readable and fill unfilled lines, ignore lines not with 00 secolds (last line)
-    local date_time open high low close lastopen lasthigh lastlow lastclose volume
-    while IFS=, read -r timestamp open high low close volume; do 
-      date_time=$(printf "%(%Y-%m-%d %H:%M:%S)T" $timestamp)
-      [ -z "$open" ] && open=$lastopen
-      [ -z "$high" ] && high=$lasthigh
-      [ -z "$low" ] && low=$lastlow
-      [ -z "$close" ] && close=$lastclose
-      [ -z "$volume" ] && volume=0
-      lastopen=$open 
-      lasthigh=$high
-      lastlow=$low
-      lastclose=$close
-      echo "$date_time,$open,$high,$low,$close,$volume"
-    done < "${f_targetcsvtmp}.unixtime" | grep ":00," >${f_targetcsvtmp}
-  fi
+  # Download data from yahoo
+  g_wget -O "${f_targetjsontmp}" "https://query1.finance.yahoo.com/v8/finance/chart/${f_item}?interval=${f_timeframe}&period1=${f_from}&period2=${f_sec}" 2>"${f_targetjsontmp}".err
+
+  # Create csv from json
+  jq -r '.chart.result[0] as $result | range(0; $result.timestamp | length) | [$result.timestamp[.], $result.indicators.quote[0].open[.], $result.indicators.quote[0].high[.], $result.indicators.quote[0].low[.], $result.indicators.quote[0].close[.], $result.indicators.quote[0].volume[.]] | @csv' "${f_targetjsontmp}" >"${f_targetcsvtmp}.unixtime" 2>"${f_targetjsontmp}".err
+
+  # remove last/open timeframe (use only closed)
+  sed -i '$d' "${f_targetcsvtmp}.unixtime"
+
+  # change unix time to human readable and fill unfilled lines, ignore lines not with 00 secolds (last line)
+  local date_time open high low close lastopen lasthigh lastlow lastclose volume
+  while IFS=, read -r timestamp open high low close volume
+  do
+    if [ "$f_timeframe" = "1d" ] || [ "$f_timeframe" = "1mo" ]
+    then
+      printf -v date_time "%(%Y-%m-%d)T" $timestamp
+    elif [ "$f_timeframe" = "1wk" ]
+    then
+      # on week 1 day back like crypto assets
+      date_time=$(date -d "yesterday $(date -d "@$timestamp" "+%Y-%m-%d")" "+%Y-%m-%d")
+    else
+      printf -v date_time "%(%Y-%m-%d %H:%M:%S)T" $timestamp
+    fi
+    [ -z "$open" ] && open=$lastopen
+    [ -z "$high" ] && high=$lasthigh
+    [ -z "$low" ] && low=$lastlow
+    [ -z "$close" ] && close=$lastclose
+    [ -z "$volume" ] && volume=0
+    lastopen=$open 
+    lasthigh=$high
+    lastlow=$low
+    lastclose=$close
+    echo "$date_time,$open,$high,$low,$close,$volume"
+  done < "${f_targetcsvtmp}.unixtime" >${f_targetcsvtmp}

  # error if no csvfile available
  if ! [ -s "${f_targetcsvtmp}" ]
@@ -129,6 +140,7 @@ function get_marketdata_yahoo {
  fi
  
  # put the csvs together
+  # history-yahoo file
  if [ -s "${f_targetcsv}" ] && [ -s "${f_targetcsvtmp}" ]
  then
    egrep -h "^[1-9][0-9][0-9][0-9]-[0-1][0-9]-[0-9][0-9].*,[0-9]" "${f_targetcsv}" "${f_targetcsvtmp}" | sort -k1,2 -t, -u | sort -k1,1 -t, -u >"${f_targetcsv}.tmp"
@@ -137,5 +149,14 @@ function get_marketdata_yahoo {
    egrep -h "^[1-9][0-9][0-9][0-9]-[0-1][0-9]-[0-9][0-9].*,[0-9]" "${f_targetcsvtmp}" | sort -k1,2 -t, -u >"$f_targetcsv"
  fi

-}
+  # bots history file
+  if [ -s "${f_targetbotcsv}" ] && [ -s "${f_targetcsv}" ]
+  then
+    egrep -h "^[1-9][0-9][0-9][0-9]-[0-1][0-9]-[0-9][0-9].*,[0-9]" "${f_targetbotcsv}" "${f_targetcsv}" | sort -k1,2 -t, -u | sort -k1,1 -t, -u >"${f_targetbotcsv}.tmp"
+    mv "${f_targetbotcsv}.tmp" "${f_targetbotcsv}"
+  else
+    egrep -h "^[1-9][0-9][0-9][0-9]-[0-1][0-9]-[0-9][0-9].*,[0-9]" "${f_targetcsv}" | sort -k1,2 -t, -u >"$f_targetbotcsv"
+  fi

+
+}
--- a/dabo/functions/get_ohlcv-candle.sh
+++ b/dabo/functions/get_ohlcv-candle.sh
@@ -35,26 +35,18 @@ function get_ohlcv-candles {
      g_echo_note "Fetching/Refreshing $f_eco_asset $f_timeframe"
      f_histfile="asset-histories/ECONOMY-${f_eco_asset}.history.${f_timeframe}.csv"

-      # 4h timefrage does not exist on coinmarketcap finance so calc from 1h
+      # 4h timeframe does not exist on yahoo finance so calc from 1h
      if [ "$f_timeframe" = "4h" ]
      then
        f_1h_histfile="asset-histories/ECONOMY-${f_eco_asset}.history.1h.csv"
        [ -s "$f_1h_histfile" ] && convert_ohlcv_1h_to_4h "$f_1h_histfile" "$f_histfile"
        f_add_missing_ohlcv_intervals "$f_histfile" 4h
-      elif [ "$f_timeframe" = "1d" ]
-      then
-        f_1h_histfile="asset-histories/ECONOMY-${f_eco_asset}.history.1h.csv"
-        [ -s "$f_1h_histfile" ] && convert_ohlcv_1h_to_1d "$f_1h_histfile" "$f_histfile"
-        f_add_missing_ohlcv_intervals "$f_histfile" 1d
-      elif [ "$f_timeframe" = "1w" ]
-      then
-        f_1d_histfile="asset-histories/ECONOMY-${f_eco_asset}.history.1d.csv"
-        [ -s "$f_1d_histfile" ] && convert_ohlcv_1d_to_1w "$f_1d_histfile" "$f_histfile"
      else
-        get_ohlcv-candle "${f_eco_asset}" ${f_timeframe} "${f_histfile}" "ECONOMY-${f_eco_asset}"
+        #get_ohlcv-candle "${f_eco_asset}" ${f_timeframe} "${f_histfile}" "ECONOMY-${f_eco_asset}"
+        get_marketdata_yahoo ${f_eco_asset} ECONOMY-${f_eco_asset} ${f_timeframe}
      fi
      # refresh latest indicators
-      [ -s "${f_histfile}" ] && get_indicators "${f_histfile}" 900
+      [ -s "${f_histfile}" ] && get_indicators "${f_histfile}" 51
    done
  done

@@ -83,7 +75,7 @@ function get_ohlcv-candles {
      get_ohlcv-candle "$f_symbol" $f_timeframe "${f_histfile}" && printf '%(%Y-%m-%d %H:%M:%S)T' >>"$f_histfile.fetched"

      # refresh latest indicators
-      get_indicators "${f_histfile}" 900 
+      get_indicators "${f_histfile}" 51

      rm -f "${f_histfile}.fetching"

@@ -118,13 +110,10 @@ function get_ohlcv-candle {
    if [[ $f_asset =~ ^ECONOMY- ]]
    then
      # economy from yahoo finance
-      if [ "$f_timeframe" = "1h" ] || [ "$f_timeframe" = "15m" ] || [ "$f_timeframe" = "5m" ]
+      if ! get_marketdata_yahoo "$f_symbol" "$f_asset" $f_timeframe
      then
-        if ! get_marketdata_yahoo "$f_symbol" "$f_asset" $f_timeframe
-        then
-          g_echo_error "$f_get_marketdata_coinmarketcap_error"
-          return 1
-        fi
+        g_echo_error "$f_get_marketdata_coinmarketcap_error"
+        return 1
      fi
      f_histfile_extdata=$f_histfile_yahoo
    else
@@ -159,6 +148,7 @@ function get_ohlcv-candle {
      g_array $f_data f_data_ref +
    else
      # from coinmarketcap/yahoo
+      
      g_array "$f_histfile_extdata" f_data_ref
    fi
 
@@ -168,10 +158,11 @@ function get_ohlcv-candle {
    g_array "${f_data_array[-1]}" f_last_data_unit_ref ,
    [ -z "$f_extdata" ] && printf -v f_last_unit_date '%(%Y-%m-%d %H:%M:%S)T' ${f_last_data_unit_ref[0]::-3}
    [ -n "$f_extdata" ] && f_last_unit_date="${f_last_data_unit_ref[0]}"
-    
-    #echo "grep -q ^\"$f_last_unit_date\" \"$f_histfile\""
+    # exit if we have already in the newest date
    [ -s "$f_histfile" ] && grep -q ^"${f_last_unit_date}," "$f_histfile" && break

+    
+
    # go through data and write to history file if new units available
    for f_data_unit in "${f_data_array[@]}"
    do
@@ -297,7 +288,7 @@ function convert_ohlcv_1h_to_4h {
    fi
 
    # Read the input file line by line
-    while IFS=',' read -r f_date f_1h_open f_1h_high f_1h_low f_1h_close f_1h_volume f_rest
+    grep -h "$f_latest_date" -A99999 "$f_input_file" | while IFS=',' read -r f_date f_1h_open f_1h_high f_1h_low f_1h_close f_1h_volume f_rest
    do
        
        # check for already converted lines
@@ -354,131 +345,135 @@ function convert_ohlcv_1h_to_4h {
        g_calc "$f_volume + $f_1h_volume"
        f_volume=$g_calc_result

-    done < "$f_input_file" >>"$f_output_file"
-}
-
-
-function convert_ohlcv_1h_to_1d {
-
-  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
-
-  local f_input_file="$1"
-  local f_output_file="$2"
-
-  local f_latestdate f_nextdate f_mytimezone f_line f_date f_open f_high f_low f_close f_volume f_inday i
-  
-  if ! [ -s "$f_input_file" ] 
-  then
-    g_echo_error "$f_input_file"
-    return 0
-  fi
-
-  # crypto timezone UTC
-  local f_target_timezone=UTC
-  # US economy timezone America/New_York
-  [[ $f_input_file =~ ECONOMY ]] && f_target_timezone="America/New_York"
- 
-  [ -s "$f_output_file" ] && f_latestdate=$(tail -n1 "$f_output_file" | cut -d, -f1)
-  [ -z "$f_latestdate" ] && f_latestdate=$(date -d "$(head -n1 "$f_input_file" | cut -d, -f1)" +%Y-%m-%d)
-  f_latestdate=$(TZ="$f_target_timezone" date -d "$f_latestdate $f_mytimezone" "+%Y-%m-%d")
-  f_nextdate=$(date -d "$f_latestdate +1day" "+%Y-%m-%d")
-  
-  #echo $f_latestdate
-  #echo $f_nextdate
-
-  # mytimezone, respecting summer/winter time
-  f_mytimezone=$(date -d "$_latestdate" +%Z) 
-
-  local f_today=$(TZ="$f_target_timezone" date "+%Y-%m-%d")
-  # check if there is a $f_latestdate
-  grep -A9999 -B24 "^$f_latestdate" "$f_input_file" >"$g_tmp/convert_ohlcv_1h_to_1d_nextlines"
-  if ! [ -s "$g_tmp/convert_ohlcv_1h_to_1d_nextlines" ] 
-  then
-    cat "$f_input_file" >"$g_tmp/convert_ohlcv_1h_to_1d_nextlines"
-    f_nextdate=$(date -d "$(head -n1 "$g_tmp/convert_ohlcv_1h_to_1d_nextlines" | cut -d, -f1)" +%Y-%m-%d)
-  fi
-
-  # go through lines and switch to $f_target_timezone
-  cat "$g_tmp/convert_ohlcv_1h_to_1d_nextlines" | grep ':00:00,' | cut -d, -f1,2,3,4,5,6 | while read f_line
-  do
-    g_array "$f_line" g_line_array ,
-    # calculate day in target timezone
-    g_line_array[0]=$(TZ="$f_target_timezone" date -d "${g_line_array[0]} $f_mytimezone" "+%Y-%m-%d")
-    [[ ${g_line_array[0]} = $f_today ]] && break
-    echo "${g_line_array[0]},${g_line_array[1]},${g_line_array[2]},${g_line_array[3]},${g_line_array[4]},${g_line_array[5]}"
-  done >"${f_output_file}.tmp" 
-  
-  # check if $f_nextdate really exists in $f_target_timezone if not add a day until it exists
-  # useful for weekends
-  i=1
-  until grep -q "^$f_nextdate" "${f_output_file}.tmp"
-  do
-    echo $f_nextdate
-    f_nextdate=$(date -d "$f_nextdate +1day" "+%Y-%m-%d")
-    i=$((i++))
-    if [ $i -gt 10 ]
-    then
-      g_echo_warn "${FUNCNAME} $@: no nextdate found after >10 iterations"
-      return 1
-    fi
-  done
-  
-  # go through converted lines
-  cat "${f_output_file}.tmp" | while read f_line
-  do
-    g_array "$f_line" g_line_array ,
-    [[ ${g_line_array[0]} = $f_today ]] && break
-
-    # wait untin next day in target file reached
-    if [[ ${g_line_array[0]} = $f_nextdate ]]
-    then
-      f_end_reached=1
-    else
-      [ -z $f_end_reached ] && continue
-    fi
-
-    # if dayend
-    if [ -n "$f_inday" ] && [[ $f_latestdate != ${g_line_array[0]} ]]
-    then
-      #echo "day end $f_date" 1>&2
-      # day end
-      f_close=${g_line_array[4]}
-      echo "$f_date,$f_open,$f_high,$f_low,$f_close,$f_volume"
-      f_inday=""
-    fi
+    done >>"$f_output_file.4htmp"
+    egrep -h "^[1-9][0-9][0-9][0-9]-[0-1][0-9]-[0-9][0-9].*,[0-9]" "$f_output_file" "$f_output_file.4htmp" | sort -k1,2 -t, -u | sort -k1,1 -t, -u >"$f_output_file.tmp"
+    mv "$f_output_file.tmp" "$f_output_file"
+    rm -f "$f_output_file.4htmp"
    
-    # calc values if inday
-    if [ -n "$f_inday" ]
-    then
-      #echo "in day $f_date" 1>&2
-      # in day
-      # add volume
-      g_calc "$f_volume+${g_line_array[5]}"
-      f_volume=$g_calc_result
-      # look for higher high
-      g_num_is_higher ${g_line_array[2]} $f_high && f_high=${g_line_array[2]}
-      # look for lower low
-      g_num_is_lower ${g_line_array[3]} $f_low && f_low=${g_line_array[3]}
-    fi
-
-    # if newday
-    if [ -z "$f_inday" ]
-    then
-      #echo "day begin ${g_line_array[0]}" 1>&2
-      # day begin
-      f_inday=1
-      f_date=${g_line_array[0]}
-      f_latestdate=$f_date
-      f_open=${g_line_array[1]}
-      f_high=${g_line_array[2]}
-      f_low=${g_line_array[3]}
-      f_volume=${g_line_array[5]}
-    fi
-
-  done >>"$f_output_file"
-
 }

+
+#function convert_ohlcv_1h_to_1d {
+#
+#  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
+#
+#  local f_input_file="$1"
+#  local f_output_file="$2"
+#
+#  local f_latestdate f_nextdate f_mytimezone f_line f_date f_open f_high f_low f_close f_volume f_inday i
+#  
+#  if ! [ -s "$f_input_file" ] 
+#  then
+#    g_echo_error "$f_input_file"
+#    return 0
+#  fi
+#
+#  # crypto timezone UTC
+#  local f_target_timezone=UTC
+#  # US economy timezone America/New_York
+#  [[ $f_input_file =~ ECONOMY ]] && f_target_timezone="America/New_York"
+# 
+#  [ -s "$f_output_file" ] && f_latestdate=$(tail -n1 "$f_output_file" | cut -d, -f1)
+#  [ -z "$f_latestdate" ] && f_latestdate=$(date -d "$(head -n1 "$f_input_file" | cut -d, -f1)" +%Y-%m-%d)
+#  f_latestdate=$(TZ="$f_target_timezone" date -d "$f_latestdate $f_mytimezone" "+%Y-%m-%d")
+#  f_nextdate=$(date -d "$f_latestdate +1day" "+%Y-%m-%d")
+#  
+#  # mytimezone, respecting summer/winter time
+#  f_mytimezone=$(date -d "$_latestdate" +%Z) 
+#
+#  local f_today=$(TZ="$f_target_timezone" date "+%Y-%m-%d")
+#  # check if there is a $f_latestdate
+#  grep -A9999 -B24 "^$f_latestdate" "$f_input_file" >"$g_tmp/convert_ohlcv_1h_to_1d_nextlines"
+#  if ! [ -s "$g_tmp/convert_ohlcv_1h_to_1d_nextlines" ] 
+#  then
+#    cat "$f_input_file" >"$g_tmp/convert_ohlcv_1h_to_1d_nextlines"
+#    f_nextdate=$(date -d "$(head -n1 "$g_tmp/convert_ohlcv_1h_to_1d_nextlines" | cut -d, -f1)" +%Y-%m-%d)
+#  fi
+#
+#  # go through lines and switch to $f_target_timezone
+#  cat "$g_tmp/convert_ohlcv_1h_to_1d_nextlines" | grep ':00:00,' | cut -d, -f1,2,3,4,5,6 | while read f_line
+#  do
+#    g_array "$f_line" g_line_array ,
+#    # calculate day in target timezone
+#    g_line_array[0]=$(TZ="$f_target_timezone" date -d "${g_line_array[0]} $f_mytimezone" "+%Y-%m-%d")
+#    [[ ${g_line_array[0]} = $f_today ]] && break
+#    echo "${g_line_array[0]},${g_line_array[1]},${g_line_array[2]},${g_line_array[3]},${g_line_array[4]},${g_line_array[5]}"
+#  done >"${f_output_file}.tmp"
+#
+#  # check if $f_nextdate really exists in $f_target_timezone if not add a day until it exists
+#  # useful for weekends
+#  i=1
+#  until grep -q "^$f_nextdate" "${f_output_file}.tmp"
+#  do
+#    #echo $f_nextdate
+#    [[ $f_nextdate = $f_today ]] && return 0
+#    f_nextdate=$(date -d "$f_nextdate +1day" "+%Y-%m-%d")
+#    i=$((i++))
+#    if [ $i -gt 10 ]
+#    then
+#      g_echo_warn "${FUNCNAME} $@: no nextdate found after >10 iterations"
+#      return 1
+#    fi
+#  done
+#  
+#  # set ent mark to store latest complete day
+#  echo END >>"${f_output_file}.tmp"
+#
+#  # go through converted lines
+#  cat "${f_output_file}.tmp" | while read f_line
+#  do
+#    g_array "$f_line" g_line_array ,
+#    [[ ${g_line_array[0]} = $f_today ]] && break
+#
+#    # wait until next day in target file reached
+#    if [[ ${g_line_array[0]} = $f_nextdate ]]
+#    then
+#      f_end_reached=1
+#    else
+#      [ -z $f_end_reached ] && continue
+#    fi
+#
+#    # if dayend
+#    if [ -n "$f_inday" ] && [[ $f_latestdate != ${g_line_array[0]} ]]
+#    then
+#      # day end
+#      echo "$f_date,$f_open,$f_high,$f_low,$f_close,$f_volume"
+#      f_inday=""
+#    fi
+#    
+#    # calc values if inday
+#    if [ -n "$f_inday" ]
+#    then
+#      #echo "in day $f_date" 1>&2
+#      # in day
+#      # add volume
+#      g_calc "$f_volume+${g_line_array[5]}"
+#      f_volume=$g_calc_result
+#      # look for higher high
+#      g_num_is_higher ${g_line_array[2]} $f_high && f_high=${g_line_array[2]}
+#      # look for lower low
+#      g_num_is_lower ${g_line_array[3]} $f_low && f_low=${g_line_array[3]}
+#    fi
+#
+#    # if newday
+#    if [ -z "$f_inday" ]
+#    then
+#      #echo "day begin ${g_line_array[0]}" 1>&2
+#      # day begin
+#      f_inday=1
+#      f_date=${g_line_array[0]}
+#      f_latestdate=$f_date
+#      f_open=${g_line_array[1]}
+#      f_high=${g_line_array[2]}
+#      f_low=${g_line_array[3]}
+#      f_close=${g_line_array[4]}
+#      f_volume=${g_line_array[5]}
+#    fi
+#
+#  done >>"$f_output_file"
+#
+#}
+
 function convert_ohlcv_1d_to_1w {

  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
@@ -512,6 +507,11 @@ function convert_ohlcv_1d_to_1w {
    IFS=',' read -r f_date f_open f_high f_low f_close f_volume f_other <<< "$f_line"
    IFS='-' read -r f_year f_month f_day <<< "$f_date"

+    [ -z "$f_high" ] && f_high=$f_open
+    [ -z "$f_low" ] && f_low=$f_open
+    [ -z "$f_close" ] && f_close=$f_open
+    [ -z "$f_volume" ] && f_volume=0
+
    # use week-number to sort day data in weeks
    f_week_number=$(date -d "$f_year-$f_month-$f_day" +%U)
    f_week_number=${f_week_number##0}
@@ -528,6 +528,7 @@ function convert_ohlcv_1d_to_1w {

    f_close_prices[$f_week_year]=$f_close
    
+    [ -z "$f_volume" ] && f_volume=0
    g_calc "${f_volume_prices[$f_week_year]:-0}+$f_volume"
    f_volume_prices[$f_week_year]=$g_calc_result
  done
@@ -535,7 +536,7 @@ function convert_ohlcv_1d_to_1w {
  # go through array(s) and write down missing week data
  for f_week_year in "${!f_open_prices[@]}"
  do
-    f_week_date=$(date -d "${f_week_year:0:4}-01-01 +$((${f_week_year:4}-1)) week" +%F)
+    f_week_date=$(date -d "${f_week_year:0:4}-01-01 +$((${f_week_year:4})) week -1day" +%F)
    # ignore if date alerady exists
    grep -q ^$f_week_date, "$f_output_file" && continue
    echo "$f_week_date,${f_open_prices[$f_week_year]},${f_high_prices[$f_week_year]},${f_low_prices[$f_week_year]},${f_close_prices[$f_week_year]},${f_volume_prices[$f_week_year]}"
@@ -664,5 +665,3 @@ function f_add_missing_ohlcv_intervals {
    cat "$g_tmp/f_add_missing_ohlcv_intervals_result" >"$f_histfile"
  fi
 }
-
-
--- a/dabo/functions/get_phemex_csv_transactions.sh
+++ b/dabo/functions/get_phemex_csv_transactions.sh
@@ -0,0 +1,63 @@
+#!/bin/bash
+
+# Copyright (c) 2022-2024 olli
+#
+# This file is part of dabo (crypto bot).
+# 
+# dabo is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# dabo is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with dabo. If not, see <http://www.gnu.org/licenses/>.
+
+
+function get_phemex_csv_transactions {
+
+  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
+
+  # PHEMEX Export format: 
+  # Time (UTC),Symbol,Exec Type,Exec. Size,Direction,Exec. Price,Order Size,Order Price,Exec Value,Fee Rate,Fee Paid,Type,"ID"
+
+  if [ -s phemex-export.csv ] 
+  then
+    # explicit long
+    cat phemex-export.csv | egrep '^[0-9].+,Trade,.+ Long,' | sort | sed 's/,Long,/,Open Long,/; s/Open Long/leverage-buy/; s/Close Long/leverage-sell/; s/ /,/g' | awk -F, '{print $1" "$2","$7","$6","$5","$13","$12",phemex,"$16","$15","$18}' >TRANSACTIONS-phemex-LONG.csv.tmp
+    # explicit short
+    cat phemex-export.csv | egrep '^[0-9].+,Trade,.+ Short,' | sort | sed 's/,Short,/,Open Short,/; s/Open Short/leverage-sell/; s/Close Short/leverage-buy/; s/ /,/g' | awk -F, '{print $1" "$2","$7","$6","$5","$13","$12",phemex,"$16","$15","$18}' >TRANSACTIONS-phemex-SHORT.csv.tmp
+    # buy/sell
+    cat phemex-export.csv | egrep '^[0-9].+,Trade,.+,Short,|^[0-9].+,Trade,.+,Long,' | sort | sed 's/Short/leverage-sell/; s/Long/leverage-buy/; s/ /,/g' | awk -F, '{print $1" "$2","$7","$6","$5","$13","$12",phemex,"$16","$15","$18}' >TRANSACTIONS-phemex.csv.tmp
+    # liquidations long
+    cat phemex-export.csv | egrep '^[0-9].+,Liquidation,.+Long,' | sort | sed 's/ /,/g' | awk -F, '{print $1" "$2",liquidation,"$6","$5","$14","$13",phemex,"$17","$16","$19}' >TRANSACTIONS-phemex-liquidations-LONG.csv.tmp
+    # liquidations short
+    cat phemex-export.csv | egrep '^[0-9].+,Liquidation,.+Short,' | sort | sed 's/ /,/g' | awk -F, '{print $1" "$2",liquidation,"$6","$5","$14","$13",phemex,"$17","$16","$19}' >TRANSACTIONS-phemex-liquidations-SHORT.csv.tmp
+    # fundingfees seeem to be included in sell
+    cat TRANSACTIONS-phemex/*.csv  | egrep 'funding' | sort -u >>TRANSACTIONS-phemex.csv.tmp
+      
+    # put together
+    local f_line f_id
+    touch "TRANSACTIONS-phemex.csv"
+    
+    cat TRANSACTIONS-phemex-LONG.csv.tmp TRANSACTIONS-phemex.csv.tmp TRANSACTIONS-phemex-liquidations-LONG.csv.tmp | sort | while read f_line
+    do
+      f_id=$(echo "$f_line" | cut -d, -f10)
+      grep -q "$f_id" "TRANSACTIONS-phemex.csv" || echo $f_line >>"TRANSACTIONS-phemex.csv"
+    done
+
+    cat TRANSACTIONS-phemex-SHORT.csv.tmp TRANSACTIONS-phemex-liquidations-SHORT.csv.tmp | sort | while read f_line
+    do
+      f_id=$(echo "$f_line" | cut -d, -f10)
+      grep -q "$f_id" "TRANSACTIONS-phemex.csv" || echo $f_line >>"TRANSACTIONS-phemex.csv"
+    done
+
+    # cleanup
+    rm -f TRANSACTIONS-phemex-LONG.csv.tmp TRANSACTIONS-phemex-SHORT.csv.tmp TRANSACTIONS-phemex.csv.tmp TRANSACTIONS-phemex-liquidations-LONG.csv.tmp TRANSACTIONS-phemex-liquidations-SHORT.csv.tmp
+  fi
+}
+
--- a/dabo/functions/get_positions.sh
+++ b/dabo/functions/get_positions.sh
@@ -42,7 +42,7 @@ function get_positions {
  jq -r "
 .[] |
 select(.entryPrice != 0) |
-.symbol + \",\" + (.collateral|tostring) + \",\" + (.entryPrice|tostring) + \",\" + .side  + \",\" + (.leverage|tostring) + \",\" + (.liquidationPrice|tostring) + \",\" + (.stopLossPrice|tostring) + \",\" + (.takeProfitPrice|tostring)
+.symbol + \",\" + (.collateral|tostring) + \",\" + (.entryPrice|tostring) + \",\" + .side  + \",\" + (.leverage|tostring) + \",\" + (.liquidationPrice|tostring) + \",\" + (.stopLossPrice|tostring) + \",\" + (.takeProfitPrice|tostring) + \",\" + (.contracts|tostring)
 " CCXT_POSITIONS_RAW >CCXT_POSITIONS

  # check for takeprofit/stoploss orders if not in CCXT output (needed for phememx and maybe more exchanges)
@@ -61,7 +61,7 @@ select(.entryPrice != 0) |
    [ "${p[${f_asset}_side]}" = "short" ] && f_action=buy
    if [[ ${p[${f_asset}_side]} =~ long|short ]]
    then
-      # search fpr stoploss and takeprofit
+      # search for stoploss and takeprofit
      f_stoploss=$(egrep "^$f_symbol,Stop,$f_action,null,0," CCXT_ORDERS | cut -d , -f9)
      f_takeprofit=$(egrep "^$f_symbol,MarketIfTouched,$f_action,null,0," CCXT_ORDERS | cut -d , -f9)
      # escape : and / for sed and edit CCXT_POSITIONS if stoploss or takeprofit order found
@@ -70,9 +70,9 @@ select(.entryPrice != 0) |
      [ -n "$f_stoploss" ] && sed -i "/^$f_symbol,.*,${p[${f_asset}_side]},/s/^\(\([^,]*,\)\{6\}\)[^,]*/\1$f_stoploss/" CCXT_POSITIONS
      [ -n "$f_takeprofit" ] && sed -i "/^$f_symbol,.*,${p[${f_asset}_side]},/s/^\(\([^,]*,\)\{7\}\)[^,]*/\1$f_takeprofit/" CCXT_POSITIONS
    fi
-    
  done
-
+  
+  return 0
 }

 function get_position_array {
@@ -145,6 +145,10 @@ function get_position_line_vars {
    f_position_takeprofit_price=${f_position_array[7]}
  fi

+  #printf -v f_position_asset_amount %.2f ${f_position_array[8]}
+  #p[${f_asset}_asset_amount]=$f_position_asset_amount
+  p[${f_asset}_asset_amount]=${f_position_array[8]}
+
  # calc pnl percentage
  if [[ $f_position_side = long ]]
  then
--- a/dabo/functions/get_symbols_ticker.sh
+++ b/dabo/functions/get_symbols_ticker.sh
@@ -87,6 +87,7 @@ function get_symbols_ticker {
    local f_ticker f_symbol f_price
    declare -Ag f_tickers_array
    declare -Ag v
+    declare -Ag vr
    for f_ticker in "${f_tickers_array_ref[@]}"
    do
      f_symbol=${f_ticker%%:*}
@@ -95,6 +96,7 @@ function get_symbols_ticker {
      f_price=${f_ticker/*,/}
      g_num_exponential2normal $f_price && f_price=$g_num_exponential2normal_result
      f_tickers_array[$f_symbol]=$f_price
+      vr[${f_symbol}_price]=$f_price
      v[${f_symbol}_price]=$f_price
    done
  fi
--- a/dabo/functions/get_transactions.sh
+++ b/dabo/functions/get_transactions.sh
@@ -113,7 +113,7 @@ function get_transactions {
        f_ccxt "print(${STOCK_EXCHANGE}.fetchFundingHistory('$f_symbol', limit=200, params={'paginate': True}))" && echo -n $f_ccxt_result >"${f_symbol_file}.FundingFees"
        cat ${f_symbol_file}.FundingFees | jq -r "
 .[] |
-.datetime + \",fundingfee,$f_asset,0,\" + .code  + \",0\" + \",$f_exchange,\" + .code  + \",\" + (.amount|tostring)
+.datetime + \",fundingfee,$f_asset,0,\" + .code  + \",0\" + \",$f_exchange,\" + .code  + \",\" + (.amount|tostring) 
 " >>$f_symbol_file_csv_tmp
        
        # remove the ':' in f_currency
@@ -127,46 +127,27 @@ function get_transactions {
      cat "$f_symbol_file" | jq -r "
 .[] |
 select(.side==\"buy\" or .side==\"sell\") |
- select(.info.posSide=null) |
- select(.symbol!= null) |
-.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring)
-" >>"$f_symbol_file_csv_tmp"
-
-      # get longs (posSide=="1")
-      cat "$f_symbol_file" | jq -r "
-.[] |
- select(.side==\"buy\" or .side==\"sell\") |
- select(.info.posSide=\"1\") |
- select(.symbol!= null) |
-.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring)
-" >>"$f_symbol_file_csv_tmp"
-
-      # get shorts (posSide=="2") sell first, then buy (https://github.com/ccxt/ccxt/issues/22518)
-      cat "$f_symbol_file" | jq -r "
-.[] |
- select(.side==\"buy\" or .side==\"sell\") |
- select(.info.posSide==\"2\") |
- select(.symbol!=null) |
-.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring) + \",short\"
+ select(.symbol != null) |
+ select(.type != null) |
+.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring) + \",\" +  .id
 " >>"$f_symbol_file_csv_tmp"

+#      # get longs (posSide=="1")
+#      cat "$f_symbol_file" | jq -r "
+#.[] |
+# select(.side==\"buy\" or .side==\"sell\") |
+# select(.info.posSide=\"1\") |
+# select(.symbol!= null) |
+#.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring) + \",\" +  .id
+#" >>"$f_symbol_file_csv_tmp"
+#
 #      # get shorts (posSide=="2") sell first, then buy (https://github.com/ccxt/ccxt/issues/22518)
 #      cat "$f_symbol_file" | jq -r "
 #.[] |
-# select(.side==\"sell\") |
+# select(.side==\"buy\" or .side==\"sell\") |
 # select(.info.posSide==\"2\") |
 # select(.symbol!=null) |
-# .side = \"sell\" |
-#.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring) + \",short\"
-#" >>"$f_symbol_file_csv_tmp"
-#
-#      cat "$f_symbol_file" | jq -r "
-#.[] |
-# select(.side==\"buy\") |
-# select(.info.posSide==\"2\") |
-# select(.symbol!=null) |
-# .side = \"buy\" |
-#.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring) + \",short\"
+#.datetime + \",$f_leverage\" + .side + \",$f_asset,\" + (.amount|tostring) + \",$f_currency,\" + (.cost|tostring) + \",$f_exchange,\" + .fee.currency  + \",\" +  (.fee.cost|tostring) + \",\" +  .id
 #" >>"$f_symbol_file_csv_tmp"

      if [ -s "$f_symbol_file_csv_tmp" ] 
@@ -226,14 +207,20 @@ function get_transactions {
    done
    fi

-    # put all sorted n one file
-    if [ -s TRANSACTIONS-$f_exchange.csv ] 
-    then
-      cat "TRANSACTIONS-$f_exchange/"*.csv TRANSACTIONS-$f_exchange.csv | sort -u >TRANSACTIONS-$f_exchange.csv.tmp
-      mv TRANSACTIONS-$f_exchange.csv.tmp TRANSACTIONS-$f_exchange.csv
-    else
-      cat "TRANSACTIONS-$f_exchange/"*.csv | sort -u >TRANSACTIONS-$f_exchange.csv
-    fi
+    # put all sorted in one file. duplicates check with id
+    local f_line f_date f_sym
+    touch "TRANSACTIONS-$f_exchange.csv"
+    grep -vh ,fundingfee, "TRANSACTIONS-$f_exchange/"*.csv | while read f_line
+    do
+      f_id=$(echo "$f_line" | cut -d, -f10)
+      grep -q "$f_id" "TRANSACTIONS-$f_exchange.csv" || echo $f_line >>"TRANSACTIONS-$f_exchange.csv" 
+    done
+    grep -h ,fundingfee, "TRANSACTIONS-$f_exchange/"*.csv | while read f_line
+    do
+      f_date=$(echo "$f_line" | cut -d: -f1)
+      f_sym=$(echo "$f_line" | cut -d, -f3)
+      egrep -q "^$f_date:.+,$f_sym," "TRANSACTIONS-$f_exchange.csv" || echo $f_line >>"TRANSACTIONS-$f_exchange.csv"
+    done

    # Switch sides if Fiat is in Krypto side
    f_fiats="USD EUR"
@@ -247,7 +234,7 @@ function get_transactions {
        g_echo_note "Switched some fiat/krypto sides"
        #cat TRANSACTIONS-$f_exchange.csv.tmp
        cat TRANSACTIONS-$f_exchange.csv | egrep -v ",sell,$f_fiat,|,buy,$f_fiat," >>TRANSACTIONS-$f_exchange.csv.tmp
-        cat TRANSACTIONS-$f_exchange.csv.tmp | sort >TRANSACTIONS-$f_exchange.csv
+        cat TRANSACTIONS-$f_exchange.csv.tmp >TRANSACTIONS-$f_exchange.csv
      fi
    done

--- a/dabo/functions/get_values.sh
+++ b/dabo/functions/get_values.sh
@@ -22,9 +22,19 @@ function get_values {
  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"

  local f_assets="$@"
+  local f_asset_histories
+
  f_assets=${f_assets//:$CURRENCY/}
  f_assets=${f_assets//\//}

+  for f_asset in $f_assets
+  do
+    f_asset_histories+="$f_asset "
+    f_asset_histories+="MARKETDATA_BINANCE_OPEN_INTEREST_$f_asset "
+    f_asset_histories+="MARKETDATA_BINANCE_LONG_SHORT_RATIO_TAKER_$f_asset "
+    f_asset_histories+="MARKETDATA_BINANCE_LONG_SHORT_RATIO_ACCOUNT_$f_asset "
+  done
+
  local f_eco_asset f_eco_assets f_asset f_time f_prefix f_histfile f_columns f_return f_levelsfile f_tmp_levels f_first
  
  for f_eco_asset in $ECO_ASSETS
@@ -40,25 +50,34 @@ function get_values {
  # get current prices from exchange
  get_symbols_ticker
  # get values from csv files
-  #f_first=true
-  for f_asset in $f_assets BTC${CURRENCY} $f_eco_assets
+  
+  for f_asset in $f_asset_histories\
+   BTC${CURRENCY}\
+   $f_eco_assets\
+   MARKETDATA_BINANCE_OPEN_INTEREST_BTC${CURRENCY}\
+   MARKETDATA_BINANCE_LONG_SHORT_RATIO_TAKER_BTC${CURRENCY}\
+   MARKETDATA_BINANCE_LONG_SHORT_RATIO_ACCOUNT_BTC${CURRENCY}\
+   MARKETDATA_FEAR_AND_GREED_ALTERNATIVEME\
+   MARKETDATA_FEAR_AND_GREED_CNN\
+   MARKETDATA_US_CONSUMER_PRICE_INDEX_CPI\
+   MARKETDATA_US_FED_FUNDS_RATE MARKETDATA_US_UNEMPLOYMENT_RATE
  do

    # read latest ohlcv data and indicators per timeframe to vars
    for f_time in 5m 15m 1h 4h 1d 1w
    do
-      #f_prefix="${f_asset}_${f_time}_"
-      #[ "$f_first" = "true" ] && f_prefix="${f_time}_"
+      # special on ECONOMY data
      f_prefix="${f_time}_"
      if [[ "$f_asset" =~ ^ECONOMY- ]] 
      then
        f_prefix="${f_asset}_${f_time}_"
        f_prefix=${f_prefix//-/_}
      fi
+      
+      # histfile
      f_histfile="asset-histories/${f_asset}.history.${f_time}.csv"
      if ! [ -s "$f_histfile" ]
      then
-        [ "$f_time" = "1w" ] || g_echo_warn "file $f_histfile empty or does not exist"
        f_return=1
        continue
      fi
@@ -67,7 +86,6 @@ function get_values {
    done

    # read current levels
-    #v[${f_asset}_price]=${f_tickers_array[$f_asset]}
    for f_time in 1w 1d
    do
      f_levelsfile="asset-histories/${f_asset}.history.${f_time}.csv.levels"
@@ -75,30 +93,36 @@ function get_values {
      then
        # get levels
        read -r -a f_levels <"$f_levelsfile"
-        v[${f_asset}_levels_$f_time]="${f_levels[*]}"
+        vr[${f_asset}_levels_$f_time]="${f_levels[*]}"
        
        # add current price and sort
-        f_levels+=("${v[${f_asset}_price]}")
+        f_levels+=("${vr[${f_asset}_price]}")
        oldIFS="$IFS"
        IFS=$'\n' f_levels_sorted=($(sort -n <<<"${f_levels[*]}"))
        IFS="$oldIFS"
        
        # find current price and +- one for upper lower price
        for ((i=0; i<${#f_levels_sorted[@]}; i++)); do
-          if [ "${f_levels_sorted[$i]}" = "${v[${f_asset}_price]}" ]
+          if [ "${f_levels_sorted[$i]}" = "${vr[${f_asset}_price]}" ]
          then
-            v[${f_asset}_levels_${f_time}_next_up]=${f_levels_sorted[i+1]}
-            v[${f_asset}_levels_${f_time}_next_down]=${f_levels_sorted[i-1]}
+            vr[${f_asset}_levels_${f_time}_next_up]=${f_levels_sorted[i+1]}
+            vr[${f_asset}_levels_${f_time}_next_down]=${f_levels_sorted[i-1]}
            break
          fi
        done
      fi
    done
-    
-    #unset f_first
-  
  done

+  # use reverse as default to be 0 latest, 1 pre latest,...
+  unset v
+  declare -Ag v
+  for key in "${!vr[@]}"; do
+    v[$key]=${vr[$key]}
+  done
+  unset vr
+
+  # write values file for overview
  for i in "${!v[@]}"
  do 
    echo "\${v[$i]}=${v[$i]}"
--- a/dabo/functions/order.sh
+++ b/dabo/functions/order.sh
@@ -24,50 +24,89 @@ function order {
 
  # needed vars
  local f_symbol=$1
-  local f_amount=$2  # amount in $CURRENCY / if crypto_amount:XXX then amount in crypto
+  local f_amount=$2  # amount in $CURRENCY / if asset_amount:XXX then amount in invested asset
  local f_side=$3                   # buy/sell long/short
-  local f_price=$4                  # price for limit order - if not given do market order
+  local f_price=$4                  # price for limit order - if "0" do market order - "stoploss" for pure StopLoss Order and "takeprofit" for pure TakeProfit Order
  local f_stoploss=$5
  local f_takeprofit=$6
  local f_params="params={"
-  local f_type
-
+  local f_type f_side_opposite f_pos_side f_side_opposide f_trigger_sl f_trigger_tp

  ### validity checks ###
- 
+
+  if [ -z "$f_symbol" ] || [ -z "$f_amount" ] || [ -z "$f_side" ] || [ -z "$f_price" ]
+  then
+    g_echo_error "Missing values!
+Usage: order symbol amount side price [stoploss] [takeprofit]
+Given: ${FUNCNAME} $@"
+    return 1
+  fi
+
  # check symbol XXX/$CURRENCY[:$CURRENCY]
  [[ $f_symbol =~ /$CURRENCY ]] || return 1

  # check side
-  [ "$f_side" = "long" ] && f_side="buy"
-  [ "$f_side" = "short" ] && f_side="sell"
+  if [ "$f_side" = "long" ] || [ "$f_side" = "buy" ]
+  then
+    f_side="buy"
+    f_pos_side="Long"
+    f_side_opposide="sell"
+    f_trigger_sl="down"
+    f_trigger_tp="up"
+  fi
+  if [ "$f_side" = "short" ] || [ "$f_side" = "sell" ]
+  then
+    f_side="sell"
+    f_pos_side="Short"
+    f_side_opposide="buy"
+    f_trigger_sl="up"
+    f_trigger_tp="down"
+  fi
  [[ $f_side =~ ^buy$|^sell$ ]] || return 1

  # check order type limit/market
-  if [ -z "$f_price" ]
+  if [[ $f_price = 0 ]]
  then
    f_type="market"
-    f_price=0
+  elif [[ $f_price = stoploss ]]
+  then
+    f_type="stoploss" 
+    f_price="None"
+  elif [[ $f_price = takeprofit ]]
+  then
+    f_type="takeprofit"
+    f_price="None"
  else
    f_type="limit"
  fi
  
-  ### validity checks end###
+  ### validity checks end ###

-  # get amount in crypto asset
-  if [[ $f_amount =~ ^crypto_amount: ]]
+
+  # get amount in target asset
+  if [[ $f_amount =~ ^asset_amount: ]] 
  then
-    # if given in crypto
-    f_amount=${f_amount//crypto_amount:}
+    # if given in target
+    f_amount=${f_amount//asset_amount:}
  else
-    # if given in $CURRENCY
-    local f_asset=${f_symbol///*}
-    currency_converter $f_amount $CURRENCY $f_asset || return 1
-    local f_amount=$f_currency_converter_result
-  fi  
+    # on market order use current price
+    if [[ $f_type = market ]]
+    then
+      # if given in $CURRENCY
+      local f_asset=${f_symbol///*}
+      currency_converter $f_amount $CURRENCY $f_asset || return 1
+      f_amount=$f_currency_converter_result
+    # on limit order use limit price
+    elif [[ $f_type = limit ]]
+    then
+      g_calc "1/${f_price}*${f_amount}"
+      f_amount=$g_calc_result
+    fi
+  fi
+

  # check for swap/margin trades
-  if [ -n "$LEVERAGE" ] 
+  if [ -n "$LEVERAGE" ]
  then
    # do some margin things

@@ -80,51 +119,80 @@ function order {
    # set leverage
    f_ccxt "$STOCK_EXCHANGE.setLeverage($LEVERAGE, '$f_symbol')" || return 1

-    # define margibn mode isolated/cross
+    # define margin mode isolated/cross
+    #[[ $f_type =~ limit|market ]] && 
    f_params="${f_params}'marginMode': '$MARGIN_MODE', "
-
+ 
+    # calculate amount with leverage
+    if [[ $f_type != stoploss ]] && [[ $f_type != takeprofit ]]
+    then
+      g_calc "${f_amount}*${LEVERAGE}"
+      f_amount=$g_calc_result
+    fi
  else
    # short/sell not possible in spot market
    [[ $f_side =~ ^sell$ ]] || return 1
  fi

-  # Add stoploos and take profit if available
+
+  # Add stoploss and take profit if available
  if [ -n "$f_stoploss" ]
  then
-    # check for long
-    if [[ $f_side = buy ]] && g_num_is_higher_equal $f_stoploss $f_price
+    if [[ $f_type = limit ]]
    then
-      g_echo_warn "Long Order not possible: Stoploss ($f_stoploss) higher then buy price ($f_price)"
-      return 1
+      # check for long
+      if [[ $f_side = buy ]] && g_num_is_higher_equal $f_stoploss $f_price
+      then
+        g_echo_warn "Long Order not possible: Stoploss ($f_stoploss) higher then buy price ($f_price)"
+        return 1
+      fi
+      # check for short
+      if [[ $f_side = sell ]] && g_num_is_lower_equal $f_stoploss $f_price 
+      then
+        g_echo_warn "Short Order not possible: Stoploss ($f_stoploss) lower then buy price ($f_price)"
+        return 1
+      fi 
    fi
-    # check for short
-    if [[ $f_side = sell ]] && g_num_is_lower_equal $f_stoploss $f_price 
-    then
-      g_echo_warn "Short Order not possible: Stoploss ($f_stoploss) lower then buy price ($f_price)"
-      return 1
-    fi 
    f_ccxt "print($STOCK_EXCHANGE.priceToPrecision('${f_symbol}', ${f_stoploss}))"
    f_stoploss=$f_ccxt_result
-    f_params="${f_params}'stopLossPrice': '$f_stoploss', "
+    # market or limit order with stoploss
+    if [[ $f_type =~ limit|market ]] 
+    then
+      f_params="${f_params}'stopLoss': { 'triggerPrice': $f_stoploss, 'type': 'market'  }, "
+    # stoploss (change) for open position
+    elif [[ $f_type = "stoploss" ]] 
+    then
+      f_type="market"
+      f_price=$f_stoploss
+      f_params="${f_params}'reduceOnly': True, 'triggerPrice': $f_stoploss, 'triggerDirection': '$f_trigger_sl', "
+    fi
  fi
-  if [ -n "$f_takeprofit" ] 
+  if [ -n "$f_takeprofit" ]
  then
    # check for long
-    if [[ $f_side = buy ]] && g_num_is_lower_equal $f_takeprofit $f_price
+    if [[ $f_type = limit ]]
    then
-      g_echo_warn "Long Order not possible:TakeProfit ($f_takeprofit) lower then buy price ($f_price)"
-      return 1
-    fi 
-    # check for short
-    if [[ $f_side = sell ]] && g_num_is_higher_equal $f_takeprofit $f_price
-    then
-      g_echo_warn "Short Order not possible:TakeProfit ($f_takeprofit) higher then buy price ($f_price)"
-      return 1
-    fi 
-
+      if [[ $f_side = buy ]] && g_num_is_lower_equal $f_takeprofit $f_price
+      then
+        g_echo_warn "Long Order not possible:TakeProfit ($f_takeprofit) lower then buy price ($f_price)"
+        return 1
+      fi 
+      # check for short
+      if [[ $f_side = sell ]] && g_num_is_higher_equal $f_takeprofit $f_price
+      then
+        g_echo_warn "Short Order not possible:TakeProfit ($f_takeprofit) higher then buy price ($f_price)"
+        return 1
+      fi 
+    fi
    f_ccxt "print($STOCK_EXCHANGE.priceToPrecision('${f_symbol}', ${f_takeprofit}))"
    f_takeprofit=$f_ccxt_result
-    f_params="${f_params}'takeProfitPrice': '$f_takeprofit', "
+    [[ $f_type =~ limit|market ]] && f_params="${f_params}'takeProfit': { 'triggerPrice': $f_takeprofit, 'type': 'limit', 'price': $f_takeprofit, }, "
+    if [[ $f_type = "takeprofit" ]] 
+    then
+      f_type="limit"
+      f_price=$f_takeprofit
+      f_params="${f_params}'reduceOnly': True, 'triggerPrice': $f_takeprofit, 'triggerDirection': '$f_trigger_tp', "
+    fi
  fi
 
  # end up params syntax with "}"
@@ -133,11 +201,19 @@ function order {
  # calculate price amount precision
  f_ccxt "print($STOCK_EXCHANGE.amountToPrecision('${f_symbol}', ${f_amount}))"
  f_amount=$f_ccxt_result
-  f_ccxt "print($STOCK_EXCHANGE.priceToPrecision('${f_symbol}', ${f_price}))"
-  f_price=$f_ccxt_result
+  if [[ $f_type = limit ]]
+  then
+    f_ccxt "print($STOCK_EXCHANGE.priceToPrecision('${f_symbol}', ${f_price}))"
+    f_price=$f_ccxt_result
+  fi

  # do the order
-  local f_order="symbol='${f_symbol}', type='$f_type', price=$f_price, amount=${f_amount}, side='${f_side}', ${f_params}"
+  # market order with or without stoploss/takeprofit
+  [[ $f_type = limit ]] && local f_order="symbol='${f_symbol}', type='$f_type', price=$f_price, amount=${f_amount}, side='${f_side}', ${f_params}"
+  [[ $f_type = market ]] && local f_order="symbol='${f_symbol}', type='$f_type', amount=${f_amount}, side='${f_side}', ${f_params}"
+
+  # takeprofit/stoploss only
+  [[ $f_params =~ reduceOnly ]] && local f_order="symbol='${f_symbol}', type='$f_type', amount=${f_amount}, side='${f_side_opposide}', price=$f_price, ${f_params}"
  echo "$f_order" | notify.sh -s "ORDER"
  f_ccxt "print($STOCK_EXCHANGE.createOrder(${f_order}))" || return 1

--- a/dabo/functions/order_cancel.sh
+++ b/dabo/functions/order_cancel.sh
@@ -41,17 +41,55 @@ function order_cancel {
    g_echo_note "No orders for $f_symbol/$f_asset found"
    return 0
  fi
-  
-
-#  for f_order in "${f_get_orders_array[@]}"
-#  do
-#    get_order_line_vars "$f_order"
-#    if [[ $f_symbol = $f_order_symbol ]]
-#    then
-#      f_ccxt "print(${STOCK_EXCHANGE}.cancelAllOrders('$f_symbol'))"
-#      get_orders "$f_symbol"
-#      get_orders_array
-#    fi
-#  done
-
 }
+
+
+function order_cancel_all {
+  # Info for log
+  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
+
+  local f_order
+
+  get_symbols_ticker
+  get_orders_array
+
+  for f_order in "${f_get_orders_array[@]}"
+  do
+    get_order_line_vars "$f_order"
+    if [[ $f_symbol = $f_order_symbol ]]
+    then
+      f_ccxt "print(${STOCK_EXCHANGE}.cancelAllOrders('$f_symbol'))"
+      get_orders "$f_symbol"
+      get_orders_array
+    fi
+  done
+}
+
+function order_cancel_id {
+  # Info for log
+  g_echo_note "RUNNING FUNCTION ${FUNCNAME} $@"
+
+  local f_symbol=$1
+  local f_id=$2
+  local f_order
+
+  get_symbols_ticker
+  get_orders "$f_symbol"
+  get_orders_array
+
+  local f_asset=${f_symbol//:$CURRENCY/}
+  f_asset=${f_asset//\//}
+
+  if grep -q "$f_asset.*$f_id" values-orders
+  then
+    f_ccxt "print(${STOCK_EXCHANGE}.cancelOrder(id='${f_id}', symbol='${f_symbol}'))"
+    get_orders "$f_symbol"
+    get_orders_array
+  else
+    g_echo_note "No orders for $f_symbol/$f_asset with id $f_id found"
+    return 1
+  fi
+}
+
+
+
--- a/dabo/functions/run_strategies.sh
+++ b/dabo/functions/run_strategies.sh
@@ -30,14 +30,14 @@ function run_strategies {
  get_symbols_ticker
  get_values ${f_symbols_array_trade[*]}

-  for f_strategy in $(find /dabo/strategies -type f -name "*strategy*" ! -name "\.*")
+  for f_strategy in $(find /dabo/strategies -type f -name "*.strategy.sh" ! -name "\.*")
  do
    if ! bash -n "${f_strategy}" >$g_tmp/strat_bash_error 2>&1
    then
      g_echo_error "Error in ${f_strategy} $(cat $g_tmp/strat_bash_error)"
      continue
    fi
-    g_echo_note "Runnign strategy ${f_strategy}"
+    g_echo_note "Running strategy ${f_strategy}"
    . "${f_strategy}" || g_echo_warn "Failed ${f_strategy}"
  done

--- a/dabo/functions/transactions_overview.sh
+++ b/dabo/functions/transactions_overview.sh
@@ -27,6 +27,7 @@ function transactions_overview {
  get_bitpanda_api_transactions
  get_justtrade_csv_transactions
  get_onetrading_csv_transactions
+  get_phemex_csv_transactions

  >ALL_TRANSACTIONS_OVERVIEW.csv.tmp
  >ALL_TRANSACTIONS_OVERVIEW_WARN.csv.tmp
@@ -34,7 +35,7 @@ function transactions_overview {

  local f_exchange f_asset f_transactions_array f_transaction f_result f_asset_quantity f_asset_quantity_sold f_currency_quantity f_currency_quantity_sold f_currency_spent f_date f_type f_asset_amount f_currency f_currency_amount f_fee_currency f_fee_amount f_sell_result f_taxable f_tax_type f_one_year_ago f_currency_amount_eur f_currency_spent_eur f_currency_quantity_sold_eur f_note f_asset_quantity_remaining f_currency_remaining f_year f_currency_spent_eur_tax f_currency_quantity_sold_eur_tax f_sell_result_percentage f_sell_result_percentage_eur

-  f_assets_per_exchange=$(egrep -h -v '^DATE,TYPE,ASSET,ASSET_AMOUNT,CURRENCY,CURRENCY_AMOUNT,EXCHANGE|^#|^$|^ +$' TRANSACTIONS-*.csv | cut -d, -f3,7 | sort -u)
+  f_assets_per_exchange=$(egrep -h -v '^DATE,TYPE,ASSET,ASSET_AMOUNT,CURRENCY,CURRENCY_AMOUNT,EXCHANGE|^#|^$|^ +$' TRANSACTIONS-*.csv | cut -d, -f3,7 | sort -u | grep JustTrade)

  for f_asset_per_exchange in ${f_assets_per_exchange}
  do
@@ -70,6 +71,7 @@ function transactions_overview {
    mapfile -t f_transactions_array < <(egrep -h -v '^DATE,TYPE,ASSET,ASSET_AMOUNT,CURRENCY,CURRENCY_AMOUNT,EXCHANGE|^#|^$|^ +$' TRANSACTIONS-*.csv | sort -u | egrep ",${f_asset},.+,.+,${f_exchange}" | sort)
    for f_transaction in "${f_transactions_array[@]}"
    do 
+      g_echo_note "Transaction: $f_transaction"
      mapfile -d, -t f_transaction_array < <(echo $f_transaction)
      f_date=${f_transaction_array[0]}
      f_type=${f_transaction_array[1]}
@@ -211,7 +213,7 @@ function transactions_overview {
            echo "$f_date,$f_exchange,$f_type,$f_asset,$f_asset_amount,$f_currency,$f_currency_amount,$f_one_year_ago,$f_currency_spent,$f_asset_quantity,$f_result,$f_sell_result,$f_tax_type,$f_taxable,$f_currency_amount_eur,$f_result_eur,$f_sell_result_eur,$f_asset_quantity_remaining,$f_note,Sell never buyed!? Spent currency on $f_asset is 0" 1>&2
            continue
          fi
-          # if sell wahats not exists!?
+          # if sell what not exists!?
          if [ $f_asset_quantity = 0 ]
          then
            #g_echo_warn "!!!!!! Sell never buyed!? Buyed asset $f_asset is 0"
@@ -239,6 +241,7 @@ function transactions_overview {
        ## Check for ended trade (asset-quantity=0 or tttt)
        # if all is sold trade ended and calculate PNL
        local f_trade_end=0
+        local f_trade_partial_end=0
        local f_dust=0
        local f_dust_eur=0
        g_calc "$f_asset_quantity_sold==$f_asset_quantity"
@@ -274,10 +277,11 @@ function transactions_overview {
          else
            g_echo_note "Tade not closed - partial sale!? Remaining $f_asset_quantity_remaining $f_asset ($f_currency_remaining $f_currency)!?" >>ALL_TRANSACTIONS_OVERVIEW.log
            f_note="Trade not closed - partial sale. Remaining $f_asset_quantity_remaining $f_asset ($f_currency_remaining $f_currency)"
+            f_trade_partial_end=1
          fi
        fi

-        if [ ${f_trade_end} -eq 1 ]
+        if [ ${f_trade_end} -eq 1 ] || [ ${f_trade_partial_end} -eq 1 ] 
        then

          echo "Buy price: $f_currency_spent $f_currency ($f_currency_spent_eur EUR)" >>ALL_TRANSACTIONS_OVERVIEW.log
@@ -303,17 +307,20 @@ function transactions_overview {
          # calculate complete result EUR
          g_calc "$f_result_eur+($f_sell_result_eur)"
          f_result_eur=$g_calc_result
-   
-          # reset vars
-          f_asset_quantity=0
-          f_asset_quantity_sold=0
-          f_asset_quantity_remaining=0
-          f_currency_spent=0
-          printf -v f_currency_spent_eur_tax %.2f $f_currency_spent_eur
-          f_currency_spent_eur=0
-          f_currency_quantity_sold=0
-          printf -v f_currency_quantity_sold_eur_tax %.2f $f_currency_quantity_sold_eur
-          f_currency_quantity_sold_eur=0
+          
+          if [ ${f_trade_end} -eq 1 ]
+          then
+            # reset vars
+            f_asset_quantity=0
+            f_asset_quantity_sold=0
+            f_asset_quantity_remaining=0
+            f_currency_spent=0
+            printf -v f_currency_spent_eur_tax %.2f $f_currency_spent_eur
+            f_currency_spent_eur=0
+            f_currency_quantity_sold=0
+            printf -v f_currency_quantity_sold_eur_tax %.2f $f_currency_quantity_sold_eur
+            f_currency_quantity_sold_eur=0
+          fi
        fi

        # at leverage always full taxable
@@ -374,6 +381,7 @@ function transactions_overview {
            ## completely tax free if over 1 year
            f_taxable=0
            f_one_year_ago="yes"
+            f_tax_type="Kauf >1 Jahr zurück"
            # reduce tax free volume
            g_calc "$f_asset_amount_tax_free-$f_asset_amount"
            f_asset_amount_tax_free=$g_calc_result
@@ -437,6 +445,8 @@ function transactions_overview {
      # 24 f_sell_result_percentage_eur
      echo "$f_date,$f_exchange,$f_type,$f_asset,$f_asset_amount,$f_currency,$f_currency_amount,$f_one_year_ago,$f_currency_spent,$f_asset_quantity,$f_result,$f_sell_result,$f_tax_type,$f_taxable,$f_currency_amount_eur,$f_result_eur,$f_sell_result_eur,$f_asset_quantity_remaining,$f_note,$f_currency_spent_eur,$f_currency_quantity_sold,$f_currency_quantity_sold_eur,$f_sell_result_percentage,$f_sell_result_percentage_eur" | tee -a ALL_TRANSACTIONS_OVERVIEW.csv.tmp >>ALL_TRANSACTIONS_OVERVIEW.log

+      echo "$f_date,$f_exchange,$f_type,$f_asset,$f_asset_amount,$f_currency,$f_currency_amount,$f_one_year_ago,$f_currency_spent,$f_asset_quantity,$f_result,$f_sell_result,$f_tax_type,$f_taxable,$f_currency_amount_eur,$f_result_eur,$f_sell_result_eur,$f_asset_quantity_remaining,$f_note,$f_currency_spent_eur,$f_currency_quantity_sold,$f_currency_quantity_sold_eur,$f_sell_result_percentage,$f_sell_result_percentage_eur"
+
      if [[ $f_type =~ sell|leverage-sell ]]
      then 
        echo -e "\n"  >>ALL_TRANSACTIONS_OVERVIEW.log
--- a/dabo/functions/webpage.sh
+++ b/dabo/functions/webpage.sh
@@ -24,6 +24,7 @@ function webpage {
  
  webpage_transactions
  get_symbols_ticker
+  charts

  # create status webpage
  echo "<html>
@@ -166,7 +167,7 @@ function webpage {
    f_sold=$(egrep "$f_exchange,.+,$f_asset" ALL_TRANSACTIONS_OVERVIEW.csv | tail -n1 | cut -d, -f22)
    if ! [ "$f_amount" = 0 ]
    then
-      currency_converter $f_amount $f_asset $TRANSFER_CURRENCY || continue
+      currency_converter "$f_amount" "$f_asset" "$TRANSFER_CURRENCY" || continue
      f_currency_amount=$f_currency_converter_result
      g_calc "$f_currency_amount+($f_sold)"
      #f_currency_amount=$g_calc_result
--- a/dabo/functions/webpage_transactions.sh
+++ b/dabo/functions/webpage_transactions.sh
@@ -27,7 +27,7 @@ function webpage_transactions {
 
 
  #echo -e "\n\n========== Total Results ===========" 
-  ##echo "Trade Result: $f_trade_result EUR" 
+  #echo "Trade Result: $f_trade_result EUR" 
  #echo "Staking Result: $f_staking_rewards EUR"
  #echo "Giveaway Result: $f_giveaway EUR"
  #echo -e "Instand Trade Bonus: $f_instant_trade_bonus EUR\n"
@@ -41,6 +41,7 @@ function webpage_transactions {
    local f_exchange_tax_type
    cat ALL_TRANSACTIONS_OVERVIEW.csv | grep "^$f_tax_year-" | cut -d, -f  2,13 | sort -u | egrep -v ',$' | grep -v "Note: " | while read f_exchange_tax_type
    do
+      #echo "$f_exchange_tax_type"
      local f_exchange=$(echo $f_exchange_tax_type | cut -d, -f1)
      local f_tax_type=$(echo $f_exchange_tax_type | cut -d, -f2)

@@ -70,14 +71,14 @@ $(cat ${g_tmp}/tax_summary_$f_exchange-$f_tax_year)
 <tr><td>Date</td><td>Type of transaction</td><td>Crypto value</td><td>Fiat value</td><td>Result</td><td>Tax type</td><td>Tax amount</td></tr>
 " >TRANSACTIONS_OVERVIEW-${f_exchange}-${f_tax_year}.html.tmp

-      cat ALL_TRANSACTIONS_OVERVIEW.csv | grep "^${f_tax_year}-" | grep ",${f_exchange}," |  awk -F, '
-{printf "<tr><td>"$1"</td><td>"$3"</td><td>"$5" "$4"</td><td>"}
-{printf("%.2f", $15)}
-{printf " EUR </td><td>"}
-{printf("%.2f", $17)}
-{printf " EUR</td><td>"$13"</td><td>"}
-{printf("%.2f", $14)}
-{print " EUR</td></tr>"}' >>TRANSACTIONS_OVERVIEW-${f_exchange}-${f_tax_year}.html.tmp
+cat ALL_TRANSACTIONS_OVERVIEW.csv | grep "^${f_tax_year}-" | grep ",${f_exchange}," | awk -F, '
+{
+    printf "<tr><td>%s</td><td>%s</td><td>%s %s</td><td>%.2f EUR</td><td>", $1, $3, $5, $4, $15
+    if ($17 != "") {
+        printf "%.2f EUR", $17
+    }
+    printf "</td><td>%s</td><td>%.2f EUR</td></tr>\n", $13, $14
+}' >>TRANSACTIONS_OVERVIEW-${f_exchange}-${f_tax_year}.html.tmp

      echo "</table></body></html>" >>TRANSACTIONS_OVERVIEW-${f_exchange}-${f_tax_year}.html.tmp
      mv TRANSACTIONS_OVERVIEW-${f_exchange}-${f_tax_year}.html.tmp ../TRANSACTIONS_OVERVIEW-${f_exchange}-${f_tax_year}.html
@@ -85,7 +86,7 @@ $(cat ${g_tmp}/tax_summary_$f_exchange-$f_tax_year)
      

    done
-    echo ""
+    #echo ""
  done

 }
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -166,7 +166,7 @@ services:
      resources:
        limits:
          cpus: '1'
-          memory: 1024M
+          memory: 512M

  dabo-calc-levels:
    build:
--- a/strategies/buy.example.conf
+++ b/strategies/buy.example.conf
@@ -1,56 +0,0 @@
-
-# remove this line if you are fine with this settings and want to use them on your own risk!!!
-# It is strongly recommended to test this strategy first via analyze.sh!!!
-return 1
-
-
-# GOOD_MARKET_PERFORMANCE_INDEX defines from which growth the market is considered good/favorable for investment.
-# The market performance is calculated from the average percentage development of various indicators such as development MSCI World, Bitcoin and Ethereum as well as forecasts.
-# for details see functions/market_performance.sh
-# If the market performance is  under this value no buying will be done
-local GOOD_MARKET_PERFORMANCE_INDEX="100"
-
-###### BUY CONDITIONS ######
-
-### RSI Indicator checks
-# Don't buy if the RSI-XX value is >= BUY_RSIXX_BUY_SIGNAL_UNTIL
-local BUY_RSI5_SIGNAL_UNTIL="0"
-local BUY_RSI14_SIGNAL_UNTIL="0"
-local BUY_RSI21_SIGNAL_UNTIL="0"
-local BUY_RSI60_SIGNAL_UNTIL="0"
-local BUY_RSI120_SIGNAL_UNTIL="0"
-local BUY_RSI240_SIGNAL_UNTIL="0"
-local BUY_RSI420_SIGNAL_UNTIL="0"
-local BUY_RSI720_SIGNAL_UNTIL="0"
-
-# Don't buy if the RSI-XX value is <= BUY_RSIXX_BUY_SIGNAL_FROM
-local BUY_RSI5_SIGNAL_FROM="0"
-local BUY_RSI14_SIGNAL_FROM="0"
-local BUY_RSI21_SIGNAL_FROM="0"
-local BUY_RSI60_SIGNAL_FROM="0"
-local BUY_RSI120_SIGNAL_FROM="0"
-local BUY_RSI240_SIGNAL_FROM="0"
-local BUY_RSI420_SIGNAL_FROM="0"
-local BUY_RSI720_SIGNAL_FROM="0"
-
-
-### MACD Indicator Checks
-# Don't buy if MACD Histogram Relation is not between this values.
-# The ratio is calculated by the difference between the maximum positive Histogram value (of the defined time period) and the current value.
-# Here you can specify a percentage range.
-# If only the buy signal should be considered, simply specify the range 0 and 100.
-# decimal numbers are not allowed here.
-local BUY_MACD_RELATION_FROM="0"
-local BUY_MACD_RELATION_TO="0"
-
-# Don't buy if price change is under this percentage values
-local BUY_MIN_PRICE_CHANGE_LAST_1_DAY="0.25"
-local BUY_MIN_PRICE_CHANGE_LAST_7_DAY="-1"
-local BUY_MIN_PRICE_CHANGE_LAST_14_DAY="-2"
-local BUY_MIN_PRICE_CHANGE_LAST_30_DAY="-5"
-
-
-# Dont buy if growth in the last defined time period <= BUY_MINGROWTH
-local BUY_MINGROWTH_PERIOD="30"
-local BUY_MINGROWTH="-10"
-
--- a/strategies/example.strategy.sh
+++ b/strategies/example.strategy.sh
@@ -1,9 +1,30 @@
+#!/bin/bash
+
+# Copyright (c) 2022-2024 olli
+#
+# This file is part of dabo (crypto bot).
+# 
+# dabo is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# dabo is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with dabo. If not, see <http://www.gnu.org/licenses/>.
+
+
 # Example strategy

 g_echo_note "EXAMPLE Strategy"

 ##### WARNING! This strategy is only intended as an example and should not be used with real trades. Please develop your own strategy ######
-# if you want to use ist remove the next line with return 0
+
+# if you want to use this remove the next line with return 0
 return 0

 # get vars with orders and positions
@@ -15,7 +36,8 @@ unset s_score
 unset s_score_hist
 s_score=0

-### Evaluate the traditional Market
+
+### BEGIN market scoring ###
 # correlation to crypto
 for asset in DOWJONES SP500 NASDAQ MSCIEAFE GOLD MSCIWORLD KRE
 do
@@ -23,7 +45,7 @@ do
  g_echo "scoring ECONOMY_${asset}"

  # bullish? bull market?
-  if g_num_is_higher ${v[ECONOMY_${asset}_15m_close_0]} ${v[ECONOMY_${asset}_1d_ema200_0]}
+  [ -n "${v[ECONOMY_${asset}_1d_ema200_0]}" ] && if g_num_is_higher ${v[ECONOMY_${asset}_15m_close_0]} ${v[ECONOMY_${asset}_1d_ema200_0]}
  then
    score 2 "${asset} EMA200 over last 15m close"
  else
@@ -53,7 +75,7 @@ do
  g_echo "scoring ECONOMY_${asset}"

  # bullish? bull market?
-  if g_num_is_higher ${v[ECONOMY_${asset}_15m_close_0]} ${v[ECONOMY_${asset}_1d_ema200_0]}
+  [ -n "${v[ECONOMY_${asset}_1d_ema200_0]}" ] && if g_num_is_higher ${v[ECONOMY_${asset}_15m_close_0]} ${v[ECONOMY_${asset}_1d_ema200_0]}
  then
    score -2 "${asset} EMA200 over last 15m close"
  else
@@ -76,8 +98,6 @@ do

 done

-
-
 ### Evaluate BTC and ETH
 for asset in BTC${CURRENCY} ETH${CURRENCY}
 do
@@ -85,7 +105,7 @@ do
  g_echo "scoring ${asset}"

  # bullish? bull market?
-  if g_num_is_higher ${v[${asset}_15m_close_0]} ${v[${asset}_1d_ema200_0]}
+  [ -n "${v[${asset}_1d_ema200_0]}" ] && if g_num_is_higher ${v[${asset}_15m_close_0]} ${v[${asset}_1d_ema200_0]}
  then
    score  2 "${asset} EMA200 over last 15m close"
  else
@@ -108,21 +128,28 @@ do

 done

+### END market scoring ###

+# save score until here
+market_score=$s_score
+market_score_hist=$s_score_hist

 ### Go through trading symbols
 for symbol in ${f_symbols_array_trade[@]}
 do

+  # restore market score as base
+  s_score=$market_score
+  s_score_hist=$market_score_hist
+
  asset=${symbol//:$CURRENCY/}
  asset=${asset//\//}

-
  ### Evaluate symbol
  g_echo "scoring ${asset}"

  # bullish? bull market?
-  if g_num_is_higher ${v[${asset}_15m_close_0]} ${v[${asset}_1d_ema200_0]}
+  [ -n "${v[${asset}_1d_ema200_0]}" ] && if g_num_is_higher ${v[${asset}_15m_close_0]} ${v[${asset}_1d_ema200_0]}
  then
    score  2 "${asset} EMA200 over last 15m close"
  else
@@ -177,6 +204,9 @@ do
  [[ $side = long ]] && entry_price=${v[${asset}_levels_1w_next_down]}
  [[ $side = short ]] && entry_price=${v[${asset}_levels_1w_next_up]}

+  # if entry price not given set to current price
+  [ -z "$entry_price" ] && entry_price=${v[${asset}_price]}
+
  # check for updates if order with entry price is already defined
  if [ -n "${o[${asset}_open_${side}_entry_price]}" ]
  then
@@ -227,9 +257,16 @@ do
  [[ $side = long ]] &&  g_calc "$entry_price+($entry_price/100*${takeprofitpercentage})"
  [[ $side = short ]] && g_calc "$entry_price-($entry_price/100*${takeprofitpercentage})"
  takeprofit=$g_calc_result
- 
+
+  # Use 100 of balace for trade
+  trade_balance=100
+  # altarnative calculate 2 percentage of available balace for trade
+  #get_balance
+  #g_calc "$f_CURRENCY_BALANCE/100*2"
+  #trade_balance=$g_calc_result
+
  # place the order
-  order "$symbol" 100 "$side" "$entry_price" "$stoploss" "$takeprofit"
+  order "$symbol" "$trade_balance" "$side" "$entry_price" "$stoploss" "$takeprofit"
 
 done

--- a/strategies/example_manage_positions.strategy.sh
+++ b/strategies/example_manage_positions.strategy.sh
@@ -0,0 +1,87 @@
+#!/bin/bash
+
+# Copyright (c) 2022-2024 olli
+#
+# This file is part of dabo (crypto bot).
+# 
+# dabo is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# dabo is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with dabo. If not, see <http://www.gnu.org/licenses/>.
+
+# Example strategy for managing open positions
+
+g_echo_note "EXAMPLE Strategy for managing open positions"
+
+##### WARNING! This strategy is only intended as an example and should not be used with real trades. Please develop your own strategy ######
+
+# if you want to use this remove the next line with return 0
+return 0
+
+# get vars with positions
+get_position_array
+
+# go through trading symbols
+for symbol in ${f_symbols_array_trade[@]}
+do
+  asset=${symbol//:$CURRENCY/}
+  asset=${asset//\//}
+
+  # adjust stoploss from percentage profit
+  from_profit=0.5
+  if [ -n "$LEVERAGE" ]
+  then
+    g_calc "${from_profit}*${LEVERAGE}"
+    from_profit=$g_calc_result
+  fi
+
+  # save profit by switching stoploss in profit
+  if [ -n "${p[${asset}_pnl]}" ]
+  then
+
+    # what side are we on (long or short)
+    side=${p[${asset}_side]}
+  
+    g_echo_note "Checking open $side position for $f_asset"
+
+    if g_num_is_higher ${p[${asset}_pnl_percentage]} $from_profit
+    then
+      # calculate stoploss price with half of current pnl
+      g_calc "${p[${asset}_current_price]}-((${p[${asset}_current_price]}-${p[${asset}_entry_price]})/2)"
+      stoploss_price=$g_calc_result
+   
+      # check for already existing stoploss
+      if [ -n "${o[${asset}_sl_close_${side}_id]}" ]
+      then
+        
+        # do nothing if current stoploss price is already larger/equal then half of current pnl
+        if [[ $side = long ]]
+        then
+          g_num_is_higher_equal ${o[${asset}_sl_close_${side}_stopprice]} $stoploss_price && continue
+        fi
+        if [[ $side = short ]]
+        then
+          g_num_is_lower_equal ${o[${asset}_sl_close_${side}_stopprice]} $stoploss_price && continue
+        fi
+        
+        # cancel existing stoploss order
+        order_cancel_id "$symbol" "${o[${asset}_sl_close_${side}_id]}" || continue
+      fi
+      
+      # create new stoploss
+      g_echo_ok "==== New StopLoss in profit for $asset at $stoploss_price"
+      order "$symbol" "asset_amount:${p[${asset}_asset_amount]}" ${side} stoploss "$stoploss_price"  
+
+    fi
+  fi
+
+done
+
--- a/strategies/sell.example.conf
+++ b/strategies/sell.example.conf
@@ -1,65 +0,0 @@
-# remove this line if you are fine with this settings and want to use them on your own risk!!!
-# It is strongly recommended to test this strategy first via analyze.sh!!!
-return 1
-
-
-# GOOD_MARKET_PERFORMANCE_INDEX defines from which growth the market is considered good/favorable for investment.
-# The market performance is calculated from the average percentage development of various indicators such as development MSCI World, Bitcoin and Ethereum as well as forecasts.
-# for details see functions/market_performance.sh
-# If the market performance is  under this value no buying will be done
-local GOOD_MARKET_PERFORMANCE_INDEX="2"
-
-# Force hold if result negative expect SELL_PERCENTAGE_FROM_LAST_PURCHASE_NEGATIVE. Boolean 1 for true 0 for false.
-local SELL_HOLD_IF_RESULT_NEGATIVE="1"
-
-# If the price falls by this percentage value from the purchase price, then sell in any case
-local SELL_PERCENTAGE_FROM_LAST_PURCHASE_NEGATIVE="-10"
-
-# from here only if SELL_HOLD_IF_RESULT_NEGATIVE doesn't match
-
-# If the price falls by this percentage value from the last rate
-local SELL_IF_LAST_RATE_LOWER_THEN="-0.25"
-
-### RSI Indicator checks
-
-# SELL if the RSI-XX value is <= SELL_RSIXX_SELL_SIGNAL_UNTIL
-local SELL_RSI5_SIGNAL_UNTIL="99"
-local SELL_RSI14_SIGNAL_UNTIL="99"
-local SELL_RSI21_SIGNAL_UNTIL="99"
-local SELL_RSI60_SIGNAL_UNTIL="99"
-local SELL_RSI120_SIGNAL_UNTIL="99"
-local SELL_RSI240_SIGNAL_UNTIL="99"
-local SELL_RSI420_SIGNAL_UNTIL="99"
-local SELL_RSI720_SIGNAL_UNTIL="99"
-
-# SELL if the RSI-XX value is >= SELL_RSIXX_SELL_SIGNAL_FROM
-local SELL_RSI5_SIGNAL_FROM="90"
-local SELL_RSI14_SIGNAL_FROM="90"
-local SELL_RSI21_SIGNAL_FROM="90"
-local SELL_RSI60_SIGNAL_FROM="70"
-local SELL_RSI120_SIGNAL_FROM="50"
-local SELL_RSI240_SIGNAL_FROM="50"
-local SELL_RSI420_SIGNAL_FROM="50"
-local SELL_RSI720_SIGNAL_FROM="50"
-
-# If the price after this time period is lower the the trading fee, then sell if SELL_HOLD_IF_RESULT_NEGATIVE is fine with it
-local SELL_IF_LOWER_THEN_FEE_AFTER_PERIOD="7200"
-
-### MACD Indicator Checks
-# Sell if MACD Histogram relation is < this value
-# The ratio is calculated by the difference between the maximum negative Histogram value (of the defined time period) and the current value.
-# Here you can specify a percentage range.
-# If only the sell signal should be considered, simply specify 0
-# decimal numbers are not allowed here.
-local SELL_MACD_RELATION_FROM="25"
-
-### Take profit/loss
-local TAKE_PROFIT_CHECK_AFTER_POSTITIVE_RESULTS="1"
-local TAKE_PROFIT_CHECK_AT_FEE_PLUS="0.5"
-local TAKE_PROFIT_CHECK_AT_FACTOR="1.25"
-
-local TAKE_LOSS_CHECK_AFTER_NEGATIVE_RESULTS="5"
-local TAKE_LOSS_CHECK_AT_FEE_PLUS="0"
-local TAKE_LOSS_CHECK_AT_FACTOR="2"
-
-
Author	SHA1	Message	Date
olli	a2848481eb	pnl, tax, transactions	2025-03-01 15:28:50 +01:00
olli	06bfc85485	pnl, tax, transactions	2025-02-28 17:22:42 +01:00
olli	6aec58b05e	pnl, tax, transactions	2025-02-27 10:44:55 +01:00
olli	412d09cbe1	added caching	2025-02-22 16:49:50 +01:00
olli	b0c7d0192a	fix DXY only	2024-12-25 13:44:11 +01:00
olli	0652ef3a7e	fix -delete	2024-12-25 13:31:36 +01:00
olli	0181800430	cleanup	2024-12-23 22:31:12 +01:00
olli	2fb220ccfb	first calc indicators only last for the last 810 lines to speed up things	2024-12-23 16:28:38 +01:00
olli	c70713356d	fix empty level var	2024-12-20 14:55:10 +01:00
olli	4a02ad0e51	performance improvements; fixes	2024-12-20 14:27:40 +01:00
olli	b2bf0ad683	fix: array reference	2024-12-20 14:22:48 +01:00
olli	af1497c1c2	performance improvements; fixes	2024-12-20 13:05:04 +01:00
olli	099a42d214	fix: array reference	2024-12-19 22:50:39 +01:00
olli	c728795a9a	reverse array	2024-12-19 21:50:21 +01:00
olli	78f9b15413	fixes	2024-12-19 17:03:48 +01:00
olli	29e04d9533	yahoo finance	2024-12-19 15:13:04 +01:00
olli	4dc63bd6ab	autoupdate	2024-12-18 15:14:39 +01:00
olli	58c2895238	fixes/additional infos	2024-12-18 13:07:01 +01:00
olli	adb75b4e22	fixes/additional infos	2024-12-18 12:43:04 +01:00
olli	3415280a33	fixes/additional infos	2024-12-18 12:03:31 +01:00
olli	3cd119dda1	fixes/additional infos	2024-12-18 11:35:55 +01:00
olli	0ca094d353	fixes adjust sl on short position	2024-12-17 17:21:45 +01:00
olli	6c0c786c70	fixes if no EMA200 data available	2024-12-17 13:03:59 +01:00
olli	a8132e3992	fixes	2024-12-17 11:39:38 +01:00
olli	705df375c9	fixes	2024-12-17 11:37:32 +01:00
olli	5326e3f5bc	fixes tp/sl	2024-12-17 11:07:40 +01:00
olli	9f72619f51	fixes tp/sl	2024-12-16 22:30:51 +01:00
olli	b279735ae8	fixes tp/sl	2024-12-16 18:04:39 +01:00
olli	76f556b1ad	fixes tp/sl	2024-12-16 17:38:39 +01:00
olli	21125e84e5	fix typo	2024-12-16 16:40:29 +01:00
olli	0109799a4c	fix typo	2024-12-16 16:24:05 +01:00
olli	8b83d450d0	fix asset_amount:	2024-12-16 16:12:17 +01:00
olli	6b51e610ff	sl/tp	2024-12-15 15:26:03 +01:00
olli	4328807975	added contracts/asset-amount	2024-12-13 10:15:43 +01:00
olli	8f307438cf	fix	2024-12-12 16:27:27 +01:00
olli	b79313d1e4	example manage positions	2024-12-12 16:24:19 +01:00
olli	694184a25f	update fpr ccxt triggers, stoploss-change	2024-12-12 16:23:47 +01:00
olli	0a6d7264cb	functions: cancel order(s)	2024-12-12 16:23:15 +01:00
olli	f66f1934a1	fix score, example trade balance inn % from complete balance, small fixes - thanks to andrius for hint(s)	2024-12-04 15:23:51 +01:00
olli	f0c3303246	fix score, example trade balance inn % from complete balance, small fixes - thanks to andrius for hint(s)	2024-12-04 15:22:31 +01:00
olli	4935a317a6	obsolete	2024-12-03 20:59:43 +01:00
olli	5adfc2920f	fix	2024-12-03 20:35:20 +01:00
olli	c835b51245	fix memory leak	2024-12-03 12:06:10 +01:00
olli	e2be72b43b	return 0 fix	2024-11-28 14:42:07 +01:00
olli	4122d60f34	fix market	2024-11-28 10:05:54 +01:00
olli	544fdb2a7a	fix market	2024-11-28 00:24:53 +01:00
olli	e1a4bb4a46	fix marketCC	2024-11-27 23:49:09 +01:00
olli	cd55084d54	market order fix	2024-11-27 14:50:30 +01:00
olli	4e2c90fe0b	limits	2024-11-26 11:56:47 +01:00
olli	afe83c4811	use defauölt traefik pw	2024-11-25 16:53:48 +01:00
olli	a7a94fdd18	doc sources	2024-11-05 14:03:15 +01:00
olli	bae56de644	fix no num data	2024-11-05 13:54:49 +01:00
olli	5536abbd0b	cleanup	2024-11-05 13:44:20 +01:00
olli	fb0c346e2f	added MARKETDATA	2024-11-05 13:43:27 +01:00
olli	d3352c4d28	speedup with less lines	2024-11-05 12:46:00 +01:00
olli	c9cc555b42	cleanup marketdata download: FEAR_AND_GREED_ALTERNATIVEME FEAR_AND_GREED_CNN US_CONSUMER_PRICE_INDEX_CPI US_UNEMPLOYMENT_RATE US_FED_FUNDS_RATE BINANCE_LONG_SHORT_RATIO_ACCOUNT BINANCE_LONG_SHORT_RATIO_TAKER BINANCE_OPEN_INTEREST	2024-11-05 12:45:16 +01:00
olli	fa4fec677b	percentage change year for specific values like CPI, added calc OHLCV, speed ptimization, fixes	2024-11-05 12:43:24 +01:00
olli	092bf62581	fix EMA/SMA	2024-11-05 12:38:12 +01:00
olli	4f3f7fdbec	error reporting and added get_marketdata_all	2024-11-05 12:37:29 +01:00
olli	8b17a7053b	less frequent	2024-11-05 12:36:32 +01:00
olli	2cea87bce7	rm error msg fix	2024-11-05 12:35:35 +01:00
olli	bf0bb1e77e	fix . not in printf	2024-11-02 17:16:19 +01:00
olli	98d185067b	fix 1st day in month	2024-11-01 13:40:30 +01:00
olli	655c3b4032	fix exponential number notation	2024-10-21 21:11:27 +02:00
olli	edfa828c2a	day fetches 1 minute later	2024-10-21 21:08:36 +02:00
olli	c0a3e85888	fix, calculating amount and leverage	2024-10-21 21:08:12 +02:00
olli	f69d75b1b7	fix	2024-10-18 23:07:25 +02:00
olli	c3e0060347	optimations and fixes	2024-10-18 23:00:15 +02:00
olli	d71f344777	optimations and fixes	2024-10-18 23:00:11 +02:00