{bad_name, setup} error during rebar3 escriptize

[tothlac]

Hi. I've seen today this very strange error in one of my builds. When I run rebar3 compile or rebar3 eunit everything works. Only when I call rebar3 ct I have this error:

===> Running hooks for compile in app setup (/home/otp00231916/work/myapp/_build/default/lib/setup) with configuration:
===>    {provider_hooks, [{post, [escriptize]}]}.
===> Running provider: escriptize
===> Unable to run post hooks for 'compile', command '{bad_name,setup}' in namespace 'rebar_prv_escriptize' not found.
===> throw {error,
                      {rebar_hooks,
                          {bad_provider,post,compile,
                              {rebar_prv_escriptize,{bad_name,setup}}}}} [{rebar_hooks,
                                                                           run_provider_hooks_,
                                                                           6,
                                                                           [{file,
                                                                             "/home/otp00231916/work/rebar3/apps/rebar/src/rebar_hooks.erl"},
                                                                            {line,
                                                                             73}]},
                                                                          {rebar_hooks,
                                                                           run_all_hooks,
                                                                           6,
                                                                           [{file,
                                                                             "/home/otp00231916/work/rebar3/apps/rebar/src/rebar_hooks.erl"},
                                                                            {line,
                                                                             18}]},
                                                                          {rebar_prv_compile,
                                                                           finalize_compile,
                                                                           3,
                                                                           [{file,
                                                                             "/home/otp00231916/work/rebar3/apps/rebar/src/rebar_prv_compile.erl"},
                                                                            {line,
                                                                             266}]},
 ...
                                                                            []},
                                                                          {init,
                                                                           do_boot,
                                                                           3,
                                                                           []}]
===> Errors loading plugin {rplug_lib,
                                   {git,
                                    "ssh://[email protected]:7999/erfidt/rplug_lib.git",
                                    {branch,"master"}}}. Run rebar3 with DEBUG=1 set to see errors.

The plugin (rplug_lib) has been compiled before, during rebar3 compile and during eunit. All the dependencies have been already compiled, now when running rebar3 ct this error is reported.
The used version of the setup application is 2.1.0. When I go into the directory of setup , and call rebar3 compile, I don't have any problems.
rplug_lib does use setup application as a dependency.
Do you have any ideas how is this "Unable to run post hooks for 'compile', command '{bad_name,setup}' in namespace 'rebar_prv_escriptize' not found.
===> throw {error," related to being not able to load rplug_lib? What can be the problem here? I had this problem with both 3.18.0 and 3.22.1

[ferd]

The {bad_name, AppName} error is something that can be returned when the app name is looked up in code paths but is not found because it has not been built.

In this case it looks like the compile step was running, mostly completed, was running its post-hooks, one of which is "run escriptize on setup" at which point there's an error encountered around

rebar3/apps/rebar/src/rebar_prv_escriptize.erl

Lines 81 to 89 in 11c2de0

	Name ->
	AllApps = rebar_state:all_deps(State)++rebar_state:project_apps(State),
	case rebar_app_utils:find(rebar_utils:to_binary(Name), AllApps) of
	{ok, AppInfo} ->
	AppInfo;
	_ ->
	?PRV_ERROR({bad_name, Name})
	end
	end,

-- this error is because setup is configured as the main app for the escript but it is not found as fully built either within the deps or main applications.

Without access to the code it's difficult to go further, but one of my debugging steps would be to look into the ebin directory of the app within the test profile after encountering a failure. You can also run with DEBUG=1 to see if there are any issues during the compilation step.

[tothlac]

The previous log was created with DEBUG=1 option, without that I would have only

===> Errors loading plugin {rplug_lib,
                                   {git,
                                    "ssh://[email protected]:7999/erfidt/rplug_lib.git",
                                    {branch,"master"}}}. Run rebar3 with DEBUG=1 set to see errors.

which I still don't understand, setup and this plugins are not related to each other. Actually the same scenario is repeated for all other plugins. After this error report rebar3 tries to compile everything again and again, and the same error is printed , but for a different plugin.

Unfortunately I can't upload the code, because it happens on a repository containing business logic.
ebin directory contains the following:

[otp00231916@erldepupgr01 setup]$ pwd
/home/otp00231916/work/myapp/_build/test/lib/setup
[otp00231916@erldepupgr01 setup]$ ls -al ebin/
total 133
drwxr-x---. 2 otp00231916 otp00231916     9 Aug 23 13:16 .
drwxr-x---. 4 otp00231916 otp00231916    12 Aug 23 13:16 ..
-rw-r-----. 1 otp00231916 otp00231916   793 Aug 23 13:16 setup.app
-rw-r-----. 1 otp00231916 otp00231916  1492 Aug 23 13:16 setup_app.beam
-rw-r-----. 1 otp00231916 otp00231916 61036 Aug 23 13:16 setup.beam
-rw-r-----. 1 otp00231916 otp00231916 39180 Aug 23 13:16 setup_gen.beam
-rw-r-----. 1 otp00231916 otp00231916  8964 Aug 23 13:16 setup_lib.beam
-rw-r-----. 1 otp00231916 otp00231916  3288 Aug 23 13:16 setup_srv.beam
-rw-r-----. 1 otp00231916 otp00231916  1592 Aug 23 13:16 setup_sup.beam

But , as I mentioned earlier rebar3 eunit was successful, and eunit also runs with the test profile. So the two questions are:

• how is it related to being not able to build a plugin
• how is it possible it was able run eunit tests, but during a ct compiling a dependency does not work? Are there any differences between compilation before ct and eunit?

If I go into the directory of setup and call a rebar3 as test compile it works.

[ferd]

Is it possible one of the plugins or libraries has a specific hook to the ct task and triggers rebuilds out of it somehow?

[tothlac]

You were right. There is a task hooked to ct. We need it because otherwise the top level application when it starts up does not have the necessary configuration read by application:get_env.
So, that's how it is hooked:

{provider_hooks,[
                       {ct,load_apps},

and the implementation of the load_apps command is the following:

-module(rebar3_load_apps).

-include("rebar3_reload.hrl").

-export([init/1, do/1, format_error/1]).

-define(PROVIDER, load_apps).
-define(DEPS, [compile]).

%%==============================================================================
%% Public API
%%==============================================================================

-spec init(State) -> Res when
    State :: rebar_state:t(),
    Res :: {ok, rebar_state:t()}.
init(State) ->
  Provider = providers:create([
                 {namespace, rebar_state:namespace(State)},
                 {name, ?PROVIDER},
                 {module, ?MODULE},
                 {bare, true},
                 {deps, ?DEPS},
                 {example, "rebar3 load_apps"},
                 {opts, []},
                 {profiles, [test]},
                 {short_desc, "Load all apps"},
                 {desc,
                    "."}]),
  {ok, rebar_state:add_provider(State, Provider)}.

-spec do(State) -> Res when
    State :: rebar_state:t(),
    Res :: {ok, rebar_state:t()} | {error, string()}.
do(State0) ->
  case rebar3_reload_lib:is_main_app(State0) of
    true ->
      erlang:system_flag(backtrace_depth, 32),
      ?INFO("Loading applications...", []),
      {ok, State} = rebar_prv_app_discovery:do(State0),
      do_(State),
      {ok, State0};
    false ->
      {ok, State0}
  end.

-spec format_error(Error) ->  Res when
    Error :: term(),
    Res :: iolist().
format_error(Reason) ->
  io_lib:format("~p", [Reason]).

%%==============================================================================
%% Internal functions
%%==============================================================================

-spec do_(State) -> Res when
    State :: rebar_state:t(),
    Res :: ok.
do_(State) ->
  Apps = rebar3_reload_lib:get_app_names(State),
  ?DEBUG("Apps: ~p", [Apps]),
  [application:load(App) || App <- Apps],
  ok.

If I delete the {ct,load_apps}, from rebar.config I don't have the errors related to the compile of other plugins, but then there will be problems during common tests, since the config of the top level application is missing. Do you have an idea how can this very simple module cause this problem? Is it related to calling rebar_prv_app_discovery:do(State0),, or something else? How else can I get the list of applications without calling rebar_prv_app_discovery:do/1 (as I remember I had to call this otherwise the list of applications was empty, or it was missing some apps). Implementation of rebar3_reload_lib:get_app_names looks like this:

get_app_names(State) ->
    AppInfos =
        case rebar_state:current_app(State) of
            undefined ->
                rebar_state:project_apps(State);
            I ->
                [I]
        end,
    [list_to_atom(binary_to_list(rebar_app_info:name(I))) || I <- AppInfos].

[tothlac]

Unfortunately if I don't call {ok, State} = rebar_prv_app_discovery:do(State0), in the plugin the list of applications returned by the above mentioned get_app_names will contain only the top level app. In this case I don't have those errors related to plugin recompilation, but not all applications are reloaded, and because of that ct will fail.

[ferd]

if you want the full list of apps you also need to have depended on the lock task (which ct does) and fetch both rebar_state:all_deps(State) and rebar_state:project_apps(State) to get both the deps and the source apps.

There's also generally already something in the CT provider to load deps if they have any config value to apply:

rebar3/apps/rebar/src/rebar_prv_common_test.erl

Line 319 in 393c7cb

[application:load(Application) || Config <- Configs, {Application, _} <- Config],