AttributeError: type object 'Module' has no attribute '_MODULE_STACK' #901
Description
I've been trying to set up a tensorforce agent using a custom network. The source of the network has to be a keras network (tensorflow.python.keras.engine.functional.Functional, which has layers from classes like tensorflow.python.keras.layers.core.Dense, tensorflow.python.keras.layers.core.Dropout).
For complex reasons I have to use tensorflow 2.3, and therefore am using tensorforce 0.6.1, which is the latest version which is compatible with that tf version.
tensorforce.core.networks.KerasNetwork looks as though it is just the right thing for the job, i.e. for taking a tensorflow/keras network and turning into a tensorforce network. However unfortunately it was only added in 0.6.3.
I have been trying to work around this using the Keras layer, tensorforce.core.layers.Keras, which does exist in 0.6.1. I understand that what this does is take a layer from a Keras network, and convert it into a layer for a tensorforce network. So my plan was to iterate through all the layers in the original Keras network, and use this wrapper to build the tensorforce network layer-by-layer. I don't need the weights from the original network, just the architecture, so thought I could get what I need using keras_network.get_config()['layers'].
This has been raising the error mentioned in the title of this post.
For example:
tensorforce.core.layers.Keras(layer='Dense', units=10).
Investigation has shown that this is not only with tensorforce.core.layers.Keras, but other tensorforce layers. E.g. creating Dense directly:
tensorforce.core.layers.Dense(size=10)
or
tensorforce.core.layers.Activation(nonlinearity='relu')
or any other layer I've tried...
All of these give the following:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
/home/ ....
----> [1](vscode-notebook-cell:/home/gabriel/Documents/code/deepneat-cage2/deep-neuroevolution-cage2/examples/cartpole%20gym/CDN%20cartpole%20%5Btensorforce%5D.ipynb#Y242sZmlsZQ%3D%3D?line=0) tensorforce.core.layers.Activation(nonlinearity='relu')
File [~/Documents/code/miniconda38/envs/venv_py38_tf23_tensorforce/lib/python3.8/site-packages/tensorforce/core/layers/misc.py:40](https://file+.vscode-resource.vscode-cdn.net/home/gabriel/Documents/code/deepneat-cage2/deep-neuroevolution-cage2/examples/cartpole%20gym/~/Documents/code/miniconda38/envs/venv_py38_tf23_tensorforce/lib/python3.8/site-packages/tensorforce/core/layers/misc.py:40), in Activation.__init__(self, nonlinearity, name, input_spec)
39 def __init__(self, *, nonlinearity, name=None, input_spec=None):
---> 40 super().__init__(name=name, input_spec=input_spec)
42 # Nonlinearity
43 if nonlinearity not in (
44 'crelu', 'elu', 'leaky-relu', 'none', 'relu', 'selu', 'sigmoid', 'softmax', 'softplus',
45 'softsign', 'swish', 'tanh'
46 ):
File [~/Documents/code/miniconda38/envs/venv_py38_tf23_tensorforce/lib/python3.8/site-packages/tensorforce/core/layers/layer.py:42](https://file+.vscode-resource.vscode-cdn.net/home/gabriel/Documents/code/deepneat-cage2/deep-neuroevolution-cage2/examples/cartpole%20gym/~/Documents/code/miniconda38/envs/venv_py38_tf23_tensorforce/lib/python3.8/site-packages/tensorforce/core/layers/layer.py:42), in Layer.__init__(self, l2_regularization, name, input_spec)
41 def __init__(self, *, l2_regularization=None, name=None, input_spec=None):
---> 42 super().__init__(l2_regularization=l2_regularization, name=name)
44 Layer._REGISTERED_LAYERS[self.name] = self
46 self.input_spec = self.default_input_spec()
File [~/Documents/code/miniconda38/envs/venv_py38_tf23_tensorforce/lib/python3.8/site-packages/tensorforce/core/module.py:166](https://file+.vscode-resource.vscode-cdn.net/home/gabriel/Documents/code/deepneat-cage2/deep-neuroevolution-cage2/examples/cartpole%20gym/~/Documents/code/miniconda38/envs/venv_py38_tf23_tensorforce/lib/python3.8/site-packages/tensorforce/core/module.py:166), in Module.__init__(self, device, l2_regularization, name)
163 self.is_saved = None
164 self.is_initialized = None
--> 166 assert len(Module._MODULE_STACK) >= 1
167 if isinstance(Module._MODULE_STACK[-1], type):
168 assert isinstance(self, Module._MODULE_STACK[-1])
AttributeError: type object 'Module' has no attribute '_MODULE_STACK'
This seems to be happening because every layer, e.g. the Keras layer, inherits (here) from the Layer class. Layer then inherits (here) from Module. The Module class (here) in line 166 expects Module._MODULE_STACK to exist, but it doesn't.
The helpful comment there on line 156 states " # _MODULE_STACK # Initialized as part of model.init()". This initialisation can be seen in model.py, line 38.
So as a result, when creating a layer, when it tries to inherit from Module, hasn't got _MODULE_STACK (since it isn't a model) and this raises the error. I think I must be missing something because how then is one to manually create a layer?
This also prevents creating a network using tensorforce.core.networks.LayeredNetwork([tensorforce.core.layers.Dense(size=10)]), even though from the docs it looks as though that ought to work.
Question: How can I make tensorforce use a network modelled on an existing Keras network, within 0.6.1? Is there a better way than using the Keras layer wrapper to iterate through the layers? If no, how can I get this to work? Thanks!