Optional Dependencies Must be Explicitly Required

Let me demonstrate the problem. There is the ex_aws Elixir library. As the name suggests, it's a wrapper around AWS API. It has optional dependencies, in particular sweet_xml.

The first problem comes from Elixir's dependency resolution. And, I believe, many other ecosystems with the concept of optional dependencies share the trait:

Optional dependency of package A can be satisfied by indirect dependency of another package B.

For example you use soap. It has sweet_xml as a dependency. If you have both ex_aws and soap installed - ex_aws will use sweet_xml because it's installed as indirect dependency of soap:

flowchart TD
  mix-exs[Declarations in mix.exs]
  
  mix-exs --> soap
  mix-exs --> ex_aws
  
  sweet_xml["sweet_xml
  (not in mix.exs)"]
  
  soap -- requires --> sweet_xml
  ex_aws -. "resolves optional dependency" .-> sweet_xml

Then imagine you work on your codebase and realize that you no longer need soap. You do not think about AWS at all because it has no relation to the domain you're changing now. When you remove soap from mix.exs, sweet_xml is removed along with it and ex_aws starts to work differently - without sweet_xml dependency. Congratulations, you've just changed behaviour of unrelated domain! Because ex_aws has the following trait:

Presence of optional dependency changes contract of some functions.

Some ex_aws functions return a parsed data structure when an XML-parsing library is present and a mere string otherwise. Unlike previous problem, this one is really disturbing.

If you are lucky enough, you will have some test that can catch this. But there is no guarantee - this codepath can be replaced with a mock in tests and you may not have proper E2E test catching it (yet). Which is pretty common situation for AWS or client-wrapper libs.

If you have no test for this - you will find this issue in production, which means good chances of data corruption. We all have painful memories about fixing corrupted state, right?

What I want to propose is to never design like this. Do something like this on configuration level:

# in config.exs or similar
config :my_lib,
  xml: true,
  # ... rest of the config

The library should raise at compile or boot time if XML parser library is missing and xml config option is set to true.

In addition to this, function return data type MUST NOT be defined by presence of any dependencies. If you have optional XML parsing do something like:

# always returns data structure,
# raises if called when `xml: false` or not set.
MyMod.call_remote_action("blablabla") 

# always return string, does not need XML parser.
MyMod.call_remote_action("blablable", raw_response: true)

Then accidental removal of an important indirect dependency will lead to compilation or boot error. As it should be if you want to have better sleep at night.

You can find another way, for example you may not define functions that require the missing optional dependency so it also leads to compilation failure. Anyway, my main take is:

If optional dependency was used, but now disappeared, it should lead to compilation- or boot-time exception.

P. S.

Do not use AWS if you can. It's a shitty product both from software and ethical perspectives.