Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

Add sample configuration loaders for inMemory and ciab #106

Open
wants to merge 1 commit into
base: configRefactor
Choose a base branch
from

Conversation

jeffmcaffer
Copy link
Contributor

Sketched out implementations of two different config loaders that go with the PR from @patrick-steele-idem for refactoring the crawler factory. That PR should end up in this branch eventually.

This change takes a poor man's dependency injection approach (thanks @willbar). The basic idea is that a config loader creates effectively a dependency injection container full of services that make up the crawler (including the crawler itself). Each service is created by a factory (or whatever you want). The standard pattern is for the factory function to get a container of services previously created, an optional name (if provided it should add itself to the container) and an optional set of default settings.

There is also a philosophical change in that the provider code should have core defaults built in so the user does not have to spec any/many options. We'll have to document the available options for each provider but that's a good thing anyway.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants