Advanced - Adding a custom indicator
Your strategy idea needs indicators, that aren't available yet? Let's see how to create and use custom indicators in Jesse.
Tutorial for a custom indicator
In this tutorial, we will convert Elliott Wave Oscillator by Centrokomopen in new window that is originally written in Pine Script to a custom indicator usable in Jesse. The following is the original code from Tradingview:
//@version=3 study("Elliott Wave Oscillator") s2=ema(close, 5) - ema(close, 34) c_color=s2 <= 0 ? red : lime plot(s2, color=c_color, style=histogram, linewidth=2)
Now, let's start the creation of our first custom indicator:
- Create a new folder called
__init__.pyfile in project's
- Then create a new file for the actual indicator, in this case we name it:
ewo.pyfor our Elliott Wave Oscillator.
- The folder structure should look like this:
├── config.py # file where you enter your database credentials, etc ├── routes.py # file where routes are defined in ├── storage # folder containing logs, chart images, etc ├── strategies # folder where you define your strategies └── custom_indicators # folder for Jesse's custom indicator ├── __init__.py └── ewo.py
- Import the custom indicator file in
from .ewo import ewo
- Now we can start creating the actual indicator code in
import numpy as np import talib from typing import Union from jesse.helpers import get_candle_source, slice_candles def ewo(candles: np.ndarray, short_period: int = 5, long_period: int = 34, source_type="close", sequential = False) -> Union[float, np.ndarray]: """ Elliott Wave Oscillator :param candles: np.ndarray :param short_period: int - default: 5 :param long_period: int - default: 34 :param source_type: str - default: close :param sequential: bool - default: False :return: Union[float, np.ndarray] """ candles = slice_candles(candles, sequential) src = get_candle_source(candles, source_type) ewo = np.subtract(talib.EMA(src, timeperiod=short_period), talib.EMA(src, timeperiod=long_period)) if sequential: return ewo else: return ewo[-1]
- Finally, to use the indicator in a trading strategy, we add the custom_indicators as library.
from jesse.strategies import Strategy import custom_indicators as cta class Strategy01(Strategy): @property def ewo(self): return cta.ewo(self.candles, short_period=5, long_period=34, source_type="close", sequential=True)
Slicing the candles
For performance gains, it's good to slice the candles to a certain size to avoid unnecessary calculations. Thats the reason we use slice_candles(). We use the configured warmup_candles_num.
We don't do it by default if sequential=True, as Jesse doesn't now how much lookback you need from your sequential indicator. But as you know it, you can remove this condition.
Too few past data change indicator values
Some indicators are influenced by the whole range of the past data. These functions are called functions with memory. Check hereopen in new window for a good explaination. That's the reason for warm_up_candles_num changing indicator values under some conditions or variations to other implementations (like TradingView).
def slice_candles(candles: np.ndarray, sequential: bool) -> np.ndarray: warmup_candles_num = get_config('env.data.warmup_candles_num', 240) if not sequential and len(candles) > warmup_candles_num: candles = candles[-warmup_candles_num:] return candles
Accessing open, close, high, low and volume
In the tutorial above we used the helper function.
src = get_candle_source(candles, source_type). This function accepts as parameters:
and returns the corresponding candle data. That is usefull in many cases, but you can get and calculate that data directly inside the indicator yourself.
candles_open = candles[:, 1] candles_close = candles[:, 2] candles_high = candles[:, 3] candles_low = candles[:, 4] candles_volume = candles[:, 5] candles_hl2 = (candles[:, 3] + candles[:, 4]) / 2 candles_hlc3 = (candles[:, 3] + candles[:, 4] + candles[:, 2]) / 3 candles_ohlc4 = (candles[:, 1] + candles[:, 3] + candles[:, 4] + candles[:, 2]) / 4
The thing with NaN and zero
You should set indicator values, that can't be calculate to
About NaN values:
- NaN is short for “Not a Number”.
- NaN values represent undefined or unrepresentable results from certain mathematical operations.
- Mathematical operations involving a NaN will either return a NaN or raise an exception.
- Comparisons involving a NaN will return False.
What's the reasons for that? Depending on your calculation you might need N candles from the past. Because of that, you won't be able to calculate a value for the indicator at the beginning of your candle data for exactly these N candles. To avoid future problems in your strategy or calculations these should be set to
np.nan and not zero. Imagine a strategy where you enter on this condition
self.indicator_value < self.price. If you had used zero instead of NaN and the current indicator value couldn't be calculate because of missing candles from the past or another problem in your calculation, the condition would be True, even if the real indicator value would be greater or the same as price. If you had used NaN it would return False as explained above and you are safe.
The thing with length
Numpy makes calculations with arrays easy. For example you can easily create hl2 prices like that:
candles_hl2 = (candles[:, 3] + candles[:, 4]) / 2
That works because
candles[:, 3] and
candles[:, 4]have the same shape / length. That's the reason why it's important to always keep the lenght consistent. Use this to match lengthsopen in new window and read this to understand why it's important to use NaN for missing values: The thing with NaN and zero.
Jesse uses Numbaopen in new window to speed up indicator calculations. Numba works good on loops and a lot numpy functions. Check the Numba docs. Hereopen in new window you will find a usage example from Jesse's indicators.
External libraries for technical indicators and things to be aware of
There are mainly two kinds of python libraries for technical indicators: Some are Pandas based and some are Numpy based. For performance reasons Jesse uses Numpy.
Talib is a perfect match for Jesse as it uses Numpy.
import talib ema = talib.EMA(candles[:, 2], timeperiod=period)
Tulipy returns Numpy, but has two things you need to be aware of.
import tulipy zlema = tulipy.zlema(np.ascontiguousarray(candles[:, 2]), period=period) zlema_with_nan = np.concatenate((np.full((candles.shape - zlema.shape), np.nan), zlema)
- Tulipy accepts only contiguousarray. The conversion can be done with:
- The returned length of the array varies. That's connected to the problem explained in The thing with NaN and zero. Tulipy just strips the values it couldn't calculate. To stay consistent with the length of our arrays we need to add those NaN ourself:
np.concatenate((np.full((candles.shape - zlema.shape), np.nan), zlema), axis=0). This compares the lengths and adds the difference as NaN to the beginning of the indicator array.
Libraries using Pandas
There are libraries out there using pandas. To use them you need to convert Numpy to Pandas. You can use this helper functionopen in new window for the conversion. The result of the indicator needs to be then converted back to numpy. Probably that will do it: pandas.Series.to_numpyopen in new window. All that converting will cost you performance and Pandas itself is less performant than Numpy.
Try to avoid loops whenever possible. Numpy and Scipy have a lot of functions that can replace the stuff that you might want to do in a loop. Loops will make the backtest very slow. The worst would be a loop within a loop. Do some research on ways to avoid them. The Jesse forum or Stackoverflow might be a good place.
How to do a loop if you couldn't avoid it:
For this example we calculate the difference of the closing price to the closing price 10 candles ago. First we create an empty array with NaNs. (For the reason check out: The thing with NaN and zero) Then we do the loop starting with i = 10, as we need 10 past candles for this calculation to work until we reach the maximal available candle length.
close = candles[:, 2] my_indicator_from_loop = np.full_like(close, np.nan) for i in range(10, len(close)): my_indicator_from_loop[i] = close[i] - close[i-10]
Consider using Numba to speed it up.
Usefull Numpy stuff
Here we collect functions and links, that are often usefull in indicator code.
def np_shift(arr: np.ndarray, num: int, fill_value=np.nan) -> np.ndarray: result = np.empty_like(arr) if num > 0: result[:num] = fill_value result[num:] = arr[:-num] elif num < 0: result[num:] = fill_value result[:num] = arr[-num:] else: result[:] = arr return result
Numpy's Forward Fill
def np_ffill(arr: np.ndarray, axis: int = 0) -> np.ndarray: idx_shape = tuple([slice(None)] + [np.newaxis] * (len(arr.shape) - axis - 1)) idx = np.where(~np.isnan(arr), np.arange(arr.shape[axis])[idx_shape], 0) np.maximum.accumulate(idx, axis=axis, out=idx) slc = [np.arange(k)[tuple([slice(None) if dim == i else np.newaxis for dim in range(len(arr.shape))])] for i, k in enumerate(arr.shape)] slc[axis] = idx return arr[tuple(slc)]
Numpy's Sliding Window
The sliding_window_viewopen in new window is a very usefull new function of numpy for indicator calculation.
Hereopen in new window you will find a usage example from Jesse's indicators.
Make arrays the same lenght
array_with_matching_lenght = np.concatenate((np.full((candles.shape - array_with_shorter_lenght.shape), np.nan), array_with_shorter_lenght)
from jesse.helpers import same_length array_with_matching_lenght = same_length(candles, array_with_shorter_lenght)
Use Numpy's Vectorized Operations
Whenever possible you want to use VectorizedOperationsopen in new window, as they are faster.