EMQX compilation getting failed on riscv

[AchyuthAGNA5675]

i tried to compile emqx with specified erlang versions:

TRIAL 1

OTP/erlang - v25.0 ---> emqx v5.0.0
error : unsupported version

but in offcial doc and on github its given as

"EMQX 4.4 requires OTP 24. EMQX 5.0 ~ 5.3 can be built with OTP 24 or 25. EMQX 5.4 and newer can be built with OTP 24 or 25."

reference : https://github.com/emqx/emqx

TRIAL 2

OTP/erlang - v24.0 ---> emqx v5.0.0 

error: emqx# make -j12
escript: Internal error: undef
=ERROR REPORT==== 18-Apr-2024::13:12:17.287934 ===
beam/beam_load.c(162): Error loading module compile:
  This BEAM file was compiled for a later version of the run-time system than 24.
  To fix this, please recompile this module with an 24 compiler.
  (Use of opcode 180; this emulator supports only up to 176.)


=ERROR REPORT==== 18-Apr-2024::13:12:17.290549 ===
Loading of /usr/local/lib/erlang/lib/compiler-8.2/ebin/compile.beam failed: badfile

full logs are given at last.

TRIAL 3

emqx 4.4 is getting compiled with OTP-23

but i need to port newer version of emqx ..kindly help, if this is an known issue.

 emqx]# make -j12
downloading rebar3 3.18.0-emqx-1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100  944k  100  944k    0     0   245k      0  0:00:03  0:00:03 --:--:-- 1118k
escript: Internal error: undef
=ERROR REPORT==== 17-Apr-2024::21:29:54.543976 ===
beam/beam_load.c(162): Error loading module compile:
  This BEAM file was compiled for a later version of the run-time system than 24.
  To fix this, please recompile this module with an 24 compiler.
  (Use of opcode 180; this emulator supports only up to 176.)


=ERROR REPORT==== 17-Apr-2024::21:29:54.546389 ===
Loading of /usr/local/lib/erlang/lib/compiler-8.2/ebin/compile.beam failed: badfile

[{compile,forms,
  [[{attribute,{2,1},file,{"scripts/merge-config.escript",2}},
    {attribute,1,module,
     'merge-config_escript__escript__1713__360594__378919__5'},
    {attribute,0,export,[{main,1}]},
    {attribute,{11,2},mode,compile},
    {function,
     {14,1},
     main,1,
     [{clause,
       {14,1},
       [{var,{14,6},'_'}],
       [],
       [{match,
         {15,5},
         {tuple,{15,5},[{atom,{15,6},ok},{var,{15,10},'BaseConf'}]},
         {call,
          {15,22},
          {remote,{15,26},{atom,{15,22},file},{atom,{15,27},read_file}},
          [{string,{15,37},"apps/emqx_conf/etc/emqx_conf.conf"}]}},
        {match,
         {17,5},
         {var,{17,5},'Cfgs'},
         {call,
          {17,12},
          {atom,{17,12},get_all_cfgs},
          [{string,{17,25},"apps/"}]}},
        {match,
         {18,5},
         {var,{18,5},'Conf'},
         {cons,
          {18,12},
          {call,
           {19,9},
           {atom,{19,9},merge},
           [{var,{19,15},'BaseConf'},{var,{19,25},'Cfgs'}]},
          {cons,
           {20,9},
           {call,
            {20,9},
            {remote,{20,15},{atom,{20,9},io_lib},{atom,{20,16},nl}},
            []},
           {cons,
            {21,9},
            {call,
             {21,9},
             {remote,{21,15},{atom,{21,9},io_lib},{atom,{21,16},nl}},
             []},
            {cons,
             {22,9},
             {string,{22,9},"include emqx_enterprise.conf"},
             {cons,
              {23,9},
              {call,
               {23,9},
               {remote,{23,15},{atom,{23,9},io_lib},{atom,{23,16},nl}},
               []},
              {nil,{24,5}}}}}}}},
        {match,
         {25,5},
         {atom,{25,5},ok},
         {call,
          {25,10},
          {remote,{25,14},{atom,{25,10},file},{atom,{25,15},write_file}},
          [{string,{25,26},"apps/emqx_conf/etc/emqx.conf.all"},
           {var,{25,62},'Conf'}]}},
        {match,
         {27,5},
         {var,{27,5},'EnterpriseCfgs'},
         {call,
          {27,22},
          {atom,{27,22},get_all_cfgs},
          [{string,{27,35},"lib-ee/"}]}},
        {match,
         {28,5},
         {var,{28,5},'EnterpriseConf'},
         {call,
          {28,22},
          {atom,{28,22},merge},
          [{string,{28,28},[]},{var,{28,32},'EnterpriseCfgs'}]}},
        {match,
         {30,5},
         {atom,{30,5},ok},
         {call,
          {30,10},
          {remote,{30,14},{atom,{30,10},file},{atom,{30,15},write_file}},
          [{string,{30,26},"apps/emqx_conf/etc/emqx_enterprise.conf.all"},
           {var,{30,73},'EnterpriseConf'}]}}]}]},
    {function,
     {32,1},
     merge,2,
     [{clause,
       {32,1},
       [{var,{32,7},'BaseConf'},{var,{32,17},'Cfgs'}],
       [],
       [{call,
         {33,5},
         {remote,{33,10},{atom,{33,5},lists},{atom,{33,11},foldl}},
         [{'fun',
           {34,9},
           {clauses,
            [{clause,
              {34,12},
              [{var,{34,13},'CfgFile'},{var,{34,22},'Acc'}],
              [],
              [{'case',
                {35,13},
                {call,
                 {35,18},
                 {remote,
                  {35,25},
                  {atom,{35,18},filelib},
                  {atom,{35,26},is_regular}},
                 [{var,{35,37},'CfgFile'}]},
                [{clause,
                  {36,17},
                  [{atom,{36,17},true}],
                  [],
                  [{match,
                    {37,21},
                    {tuple,{37,21},[{atom,{37,22},ok},{var,{37,26},'Bin1'}]},
                    {call,
                     {37,34},
                     {remote,
                      {37,38},
                      {atom,{37,34},file},
                      {atom,{37,39},read_file}},
                     [{var,{37,49},'CfgFile'}]}},
                   {'case',
                    {38,21},
                    {call,
                     {38,26},
                     {remote,
                      {38,32},
                      {atom,{38,26},string},
                      {atom,{38,33},trim}},
                     [{var,{38,38},'Bin1'},{atom,{38,44},both}]},
                    [{clause,
                      {39,25},
                      [{bin,{39,25},[]}],
                      [],
                      [{var,{39,33},'Acc'}]},
                     {clause,
                      {40,25},
                      [{var,{40,25},'Bin2'}],
                      [],
                      [{cons,
                        {40,33},
                        {var,{40,34},'Acc'},
                        {cons,
                         {40,39},
                         {call,
                          {40,39},
                          {remote,
                           {40,45},
                           {atom,{40,39},io_lib},
                           {atom,{40,46},nl}},
                          []},
                         {cons,
                          {40,52},
                          {call,
                           {40,52},
                           {remote,
                            {40,58},
                            {atom,{40,52},io_lib},
                            {atom,{40,59},nl}},
                           []},
                          {cons,
                           {40,65},
                           {var,{40,65},'Bin2'},
                           {nil,{40,69}}}}}}]}]}]},
                 {clause,
                  {42,17},
                  [{atom,{42,17},false}],
                  [],
                  [{var,{43,21},'Acc'}]}]}]}]}},
          {var,{46,9},'BaseConf'},
          {var,{47,9},'Cfgs'}]}]}]},
    {function,
     {50,1},
     get_all_cfgs,1,
     [{clause,
       {50,1},
       [{var,{50,14},'Root'}],
       [],
       [{match,
         {51,5},
         {var,{51,5},'Apps0'},
         {op,
          {51,41},
          '--',
          {call,
           {51,13},
           {remote,{51,20},{atom,{51,13},filelib},{atom,{51,21},wildcard}},
           [{string,{51,30},"*"},{var,{51,35},'Root'}]},
          {cons,
           {51,44},
           {string,{51,45},"emqx_machine"},
           {cons,{51,61},{string,{51,61},"emqx_conf"},{nil,{51,72}}}}}},
        {match,
         {52,5},
         {var,{52,5},'Apps1'},
         {op,
          {52,30},
          '++',
          {op,
           {52,20},
           '--',
           {var,{52,14},'Apps0'},
           {cons,
            {52,24},
            {string,{52,24},"emqx"},
            {cons,
             {52,24},
             {string,{52,24},"emqx_dashboard"},
             {cons,{52,24},{string,{52,24},"emqx_authz"},{nil,{52,24}}}}}},
          {call,
           {52,33},
           {remote,{52,38},{atom,{52,33},lists},{atom,{52,39},reverse}},
           [{cons,
             {52,48},
             {string,{52,48},"emqx"},
             {cons,
              {52,48},
              {string,{52,48},"emqx_dashboard"},
              {cons,{52,48},{string,{52,48},"emqx_authz"},{nil,{52,48}}}}}]}}},
        {match,
         {53,5},
         {var,{53,5},'Dirs'},
         {lc,
          {53,12},
          {call,
           {53,13},
           {remote,{53,21},{atom,{53,13},filename},{atom,{53,22},join}},
           [{cons,
             {53,27},
             {var,{53,28},'Root'},
             {cons,{53,34},{var,{53,34},'App'},{nil,{53,37}}}}]},
          [{generate,{53,47},{var,{53,43},'App'},{var,{53,50},'Apps1'}}]}},
        {call,
         {54,5},
         {remote,{54,10},{atom,{54,5},lists},{atom,{54,11},foldl}},
         [{'fun',{54,17},{function,get_cfgs,2}},
          {nil,{54,33}},
          {var,{54,37},'Dirs'}]}]}]},
    {function,
     {56,1},
     get_all_cfgs,2,
     [{clause,
       {56,1},
       [{var,{56,14},'Dir'},{var,{56,19},'Cfgs'}],
       [],
       [{match,
         {57,5},
         {var,{57,5},'Fun'},
         {'fun',
          {57,11},
          {clauses,
           [{clause,
             {57,14},
             [{var,{57,15},'E'},{var,{57,18},'Acc'}],
             [],
             [{match,
               {58,9},
               {var,{58,9},'Path'},
               {call,
                {58,16},
                {remote,{58,24},{atom,{58,16},filename},{atom,{58,25},join}},
                [{cons,
                  {58,30},
                  {var,{58,31},'Dir'},
                  {cons,{58,36},{var,{58,36},'E'},{nil,{58,37}}}}]}},
              {call,
               {59,9},
               {atom,{59,9},get_cfgs},
               [{var,{59,18},'Path'},{var,{59,24},'Acc'}]}]}]}}},
        {call,
         {61,5},
         {remote,{61,10},{atom,{61,5},lists},{atom,{61,11},foldl}},
         [{var,{61,17},'Fun'},
          {var,{61,22},'Cfgs'},
          {call,
           {61,28},
           {remote,{61,35},{atom,{61,28},filelib},{atom,{61,36},wildcard}},
           [{string,{61,45},"*"},{var,{61,50},'Dir'}]}]}]}]},
    {function,
     {63,1},
     get_cfgs,2,
     [{clause,
       {63,1},
       [{var,{63,10},'Dir'},{var,{63,15},'Cfgs'}],
       [],
       [{'case',
         {64,5},
         {call,
          {64,10},
          {remote,{64,17},{atom,{64,10},filelib},{atom,{64,18},is_dir}},
          [{var,{64,25},'Dir'}]},
         [{clause,{65,9},[{atom,{65,9},false}],[],[{var,{66,13},'Cfgs'}]},
          {clause,
           {67,9},
           [{var,{67,9},'_'}],
           [],
           [{match,
             {68,13},
             {var,{68,13},'Files'},
             {call,
              {68,21},
              {remote,{68,28},{atom,{68,21},filelib},{atom,{68,29},wildcard}},
              [{string,{68,38},"*"},{var,{68,43},'Dir'}]}},
            {'case',
             {69,13},
             {call,
              {69,18},
              {remote,{69,23},{atom,{69,18},lists},{atom,{69,24},member}},
              [{string,{69,31},"etc"},{var,{69,38},'Files'}]},
             [{clause,
               {70,17},
               [{atom,{70,17},false}],
               [],
               [{call,
                 {71,21},
                 {atom,{71,21},try_enter_child},
                 [{var,{71,37},'Dir'},
                  {var,{71,42},'Files'},
                  {var,{71,49},'Cfgs'}]}]},
              {clause,
               {72,17},
               [{atom,{72,17},true}],
               [],
               [{match,
                 {73,21},
                 {var,{73,21},'EtcDir'},
                 {call,
                  {73,30},
                  {remote,{73,38},{atom,{73,30},filename},{atom,{73,39},join}},
                  [{cons,
                    {73,44},
                    {var,{73,45},'Dir'},
                    {cons,{73,50},{string,{73,50},"etc"},{nil,{73,55}}}}]}},
                {match,
                 {76,21},
                 {var,{76,21},'Confs'},
                 {call,
                  {76,29},
                  {remote,
                   {76,36},
                   {atom,{76,29},filelib},
                   {atom,{76,37},wildcard}},
                  [{string,{76,46},"emqx*.conf"},{var,{76,60},'EtcDir'}]}},
                {match,
                 {77,21},
                 {var,{77,21},'NewCfgs'},
                 {lc,
                  {77,31},
                  {call,
                   {77,32},
                   {remote,
                    {77,40},
                    {atom,{77,32},filename},
                    {atom,{77,41},join}},
                   [{cons,
                     {77,46},
                     {var,{77,47},'EtcDir'},
                     {cons,{77,55},{var,{77,55},'Name'},{nil,{77,59}}}}]},
                  [{generate,
                    {77,70},
                    {var,{77,65},'Name'},
                    {var,{77,73},'Confs'}}]}},
                {call,
                 {78,21},
                 {atom,{78,21},try_enter_child},
                 [{var,{78,37},'Dir'},
                  {var,{78,42},'Files'},
                  {op,
                   {78,57},
                   '++',
                   {var,{78,49},'NewCfgs'},
                   {var,{78,60},'Cfgs'}}]}]}]}]}]}]}]},
    {function,
     {82,1},
     try_enter_child,3,
     [{clause,
       {82,1},
       [{var,{82,17},'Dir'},{var,{82,22},'Files'},{var,{82,29},'Cfgs'}],
       [],
       [{'case',
         {83,5},
         {call,
          {83,10},
          {remote,{83,15},{atom,{83,10},lists},{atom,{83,16},member}},
          [{string,{83,23},"src"},{var,{83,30},'Files'}]},
         [{clause,{84,9},[{atom,{84,9},false}],[],[{var,{85,13},'Cfgs'}]},
          {clause,
           {86,9},
           [{atom,{86,9},true}],
           [],
           [{call,
             {87,13},
             {atom,{87,13},get_all_cfgs},
             [{call,
               {87,26},
               {remote,{87,34},{atom,{87,26},filename},{atom,{87,35},join}},
               [{cons,
                 {87,40},
                 {var,{87,41},'Dir'},
                 {cons,{87,46},{string,{87,46},"src"},{nil,{87,51}}}}]},
              {var,{87,55},'Cfgs'}]}]}]}]}]},
    {eof,{89,1}}],
   [report]],
  []},
 {escript,parse_and_run,3,[{file,"escript.erl"},{line,319}]},
 {escript,start,1,[{file,"escript.erl"},{line,277}]},
 {init,start_em,1,[]},
 {init,do_boot,3,[]}]
make: *** [Makefile:219: conf-segs] Error 127

[ieQu1]

Hello,

Thanks for trying to run our code on an interesting new architecture. I don't think anyone has tried this before. Getting EMQX run on riskV is probably going to be quite challenging.

First of all, Erlang doesn't support JIT compilation on RiskV: erlang/otp#7498 so the code will have to run via bytecode emulator. The performance won't be on par with AMD64 and AArch64. The good news is that the bytecode is harware-independent.

The bad news is that EMQX relies on a number of native libraries. From the top of my head I can remember the following:

• Optional ones: ssl, msquic, jq
• Mandatory: rocksdb

While QUIC and JQ may be disabled, if rocksDB NIF incompatible with RiskV, it's rather tough luck.

escript: Internal error: undef
=ERROR REPORT==== 18-Apr-2024::13:12:17.287934 ===
beam/beam_load.c(162): Error loading module compile:
This BEAM file was compiled for a later version of the run-time system than 24.
To fix this, please recompile this module with an 24 compiler.
(Use of opcode 180; this emulator supports only up to 176.)

It looks like the rebar3 binary that we use is incompatible with your OTP version and/or system. You need to compile https://github.com/emqx/rebar3 manually on your architecture and OTP version and place the resulting binary in the EMQX project root directory.

[AchyuthAGNA5675]

hi @ieQu1 ,
Thanks for the reply , i have compiled rebar3 separately but still facing same issue .
but emqx 4.4 is getting compiled with OTP - v23.0
but when i try to port higher version of emqx am facing this issue.

i tried majority of combinations of OTP version with emqx , but still the results are negative.

iI t would be a great help if you can provide which version of otp might support , with emqx v5.0 (which am currently trying to port ).

Thanks & Regards
Achyuth.S

[ieQu1]

The OTP version to build the official packages can be tracked here: https://github.com/emqx/emqx/blob/master/.github/workflows/build_slim_packages.yaml#L30

The format of the tag is 5.3-2:1.15.7-26.2.1-2-ubuntu22.04' which reads as builder_version:elixir_version-otp_version-distro`. You can search the history of this file to find what OTP version was used for each EMQX tag.

[AchyuthAGNA5675]

Great!! thankyou, i will look into this and come back.

[AchyuthAGNA5675]

Hi @ieQu1 ,
Your last solution worked great , i have successfully ported EMQX on riscv .
combination EMQX - v5.6.0 <------>OTP/erlang - 26.2.4.
Thankyou so much for the support.

[ieQu1]

Great!

[AchyuthAGNA5675]

---------------------------------------------EMQX porting updated version on RISCV ------------------------------

Porting erlang/OTP which is required for emqx to compile

$ git clone https://github.com/erlang/otp.git

$ git checkout OTP-26.2.4

$ ./configure

$ make –j$(nproc)

$ make install

Porting emqx

$ git clone https://github.com/emqx/emqx.git

$ git checkout v5.6.0

$ make –j$(nproc)

Once the build is completed , enter the below command to start emqx shell

$ _build/emqx/rel/emqx/bin/emqx console