4. Chapter 4. Example of RunnableModel and Backend¶
(or back to Chapter 3)
4.1. If you are using this file in Google Colab, this block of code can help you install sciunit from PyPI in Colab environment.¶
!pip install -q sciunit
Beside the usual model in previous sections, let’s create a model that run a Backend instance to simulate and obtain results.
Firstly, import necessary components from SciUnit package.
import sciunit, random
from sciunit import Test
from sciunit.capabilities import Runnable
from sciunit.scores import BooleanScore
from sciunit.models import RunnableModel
from sciunit.models.backends import register_backends, Backend
Let’s define subclasses of SciUnit Backend, Test, and Model.
Note that:
A SciUnit Backend subclass should implement _backend_run method.
A SciUnit Backend subclass should implement run method.
class RandomNumBackend(Backend):
'''generate a random integer between min and max'''
def set_run_params(self, **run_params):
# get min from run_params, if not exist, then 0.
self.min = run_params.get('min', 0)
# get max from run_params, if not exist, then self.min + 100.
self.max = run_params.get('max', self.min + 100)
def _backend_run(self):
# generate and return random integer between min and max.
return random.randint(self.min, self.max)
class RandomNumModel(RunnableModel):
"""A model that always produces a constant number as output."""
def run(self):
self.results = self._backend.backend_run()
class RangeTest(Test):
"""Tests if the model predicts the same number as the observation."""
# Default Runnable Capability for RunnableModel
required_capabilities = (Runnable,)
# This test's 'judge' method will return a BooleanScore.
score_type = BooleanScore
def generate_prediction(self, model):
model.run()
return model.results
def compute_score(self, observation, prediction):
score = BooleanScore(
observation['min'] <= prediction and observation['max'] >= prediction
)
return score
Let’s define the model instance named model 1.
model = RandomNumModel("model 1")
We must register any backend isntance in order to use it in model instances.
set_backend and set_run_params methods can help us to set the run-parameters in the model and its backend.
register_backends({"Random Number": RandomNumBackend})
model.set_backend("Random Number")
model.set_run_params(min=1, max=10)
Next, create an observation that requires the generated random integer between 1 and 10 and a test instance that use the observation and against the model
Then we get a more quantitative summary of the results:
observation = {'min': 1, 'max': 10}
oneToTenTest = RangeTest(observation, "test 1")
score = oneToTenTest.judge(model)
print the score, and we can see the result.
print(score)
Pass