Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R abort when transcribing #69

Open
chaoliu-cl opened this issue Oct 3, 2024 · 10 comments
Open

R abort when transcribing #69

chaoliu-cl opened this issue Oct 3, 2024 · 10 comments

Comments

@chaoliu-cl
Copy link

I encountered an R abort when performing transcription, which I suspect may be related to memory leaks. This issue occurs in versions 0.4.1, 0.4, 0.3.3, 0.3.2, and 0.3.1, but everything works fine with versions 0.3 and 0.2.2. I am running this on a Windows system. Below is my sessionInfo():

R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22631)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] audio.whisper_0.3.1
@jwijffels
Copy link
Contributor

jwijffels commented Oct 3, 2024

Can you provide the wav file and the code that you use to transcribe.
As well as the code and logs of the installation of the audio.whisper package

@chaoliu-cl
Copy link
Author

I was just using the sample audio file and the example code:

audio <- system.file(package = "audio.whisper", "samples", "jfk.wav")
trans <- predict(model, newdata = audio, language = "en", n_threads = 2)

@jwijffels
Copy link
Contributor

You need to show as well the the code and logs of the installation of the audio.whisper package.
The compilation instruction that are shown in the log are important as the end of the README says:

Note that if your hardware does not support these compilation flags, you'll get a crash when transcribing audio.

Maybe you installed the package with compilation settings which are not applicable to your machine.

@chaoliu-cl
Copy link
Author

The installation log is rather long but here it is below. Do you see any incompatibilities there?

> remotes::install_github("bnosac/audio.whisper", ref = "0.3.3")
Using GitHub PAT from the git credential store.
Downloading GitHub repo bnosac/[email protected]
── R CMD build ──────────────────────────────────────────────────────────────────────────────────
✔  checking for file 'C:\Users\psych\AppData\Local\Temp\RtmpqcYmyQ\remotes34742f7641f9\bnosac-audio.whisper-f30c8b3/DESCRIPTION' ...
─  preparing 'audio.whisper': (719ms)
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building 'audio.whisper_0.3.3.tar.gz' (621ms)
   
Installing package into ‘C:/Users/psych/AppData/Local/R/win-library/4.4’
(as ‘lib’ is unspecified)
* installing *source* package 'audio.whisper' ...
** using staged installation
** libs
which: no nvcc in (/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/PROGRA~1/R/R-44~1.1/bin/x64:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/Program Files/R/R-4.4.1/bin/x64:/c/Program Files (x86)/Common Files/Oracle/Java/java8path_target_41628812:/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/c/Windows/System32:/c/Windows:/c/Windows/System32/wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files/nodejs:/c/ProgramData/chocolatey/bin:/c/Program Files/MATLAB/R2023b/bin:/c/Users/psych/anaconda3:/c/Users/psych/anaconda3/Scripts:/c/Users/psych/anaconda3/Library/bin:/c/Program Files/Git/cmd:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/Program Files/R/R-4.4.1/bin/x64:/c/Program Files (x86)/Common Files/Oracle/Java/java8path_target_41628812:/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/c/Windows/System32:/c/Windows:/c/Windows/System32/wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files/nodejs:/c/ProgramData/chocolatey/bin:/c/Program Files/MATLAB/R2023b/bin:/c/Users/psych/anaconda3:/c/Users/psych/anaconda3/Scripts:/c/Users/psych/anaconda3/Library/bin:/c/Program Files/Git/cmd:/c/Users/psych/AppData/Local/Programs/Python/Launcher:/c/Users/psych/AppData/Local/Microsoft/WindowsApps:/c/texlive/2022/bin/win32:/c/Users/psych/AppData/Local/Programs/Microsoft VS Code/bin:/c/Users/psych/AppData/Roaming/npm:/c/Users/psych/Downloads/hugo_0.115.3_windows-amd64:/d/Downloads/flutter_projects/src/flutter/bin:/c/Program Files/RStudio/resources/app/bin/quarto/bin:/c/Program Files/RStudio/resources/app/bin/postback:/c/Users/psych/AppData:/c/Program Files/RStudio/resources/app/bin/quarto/bin:/c/Program Files/RStudio/resources/app/bin/postback/)
using C++ compiler: 'G__~1.EXE (GCC) 13.2.0'
using C++11
which: no nvcc in (/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/PROGRA~1/R/R-44~1.1/bin/x64:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/Program Files/R/R-4.4.1/bin/x64:/c/Program Files (x86)/Common Files/Oracle/Java/java8path_target_41628812:/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/c/Windows/System32:/c/Windows:/c/Windows/System32/wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files/nodejs:/c/ProgramData/chocolatey/bin:/c/Program Files/MATLAB/R2023b/bin:/c/Users/psych/anaconda3:/c/Users/psych/anaconda3/Scripts:/c/Users/psych/anaconda3/Library/bin:/c/Program Files/Git/cmd:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/Program Files/R/R-4.4.1/bin/x64:/c/Program Files (x86)/Common Files/Oracle/Java/java8path_target_41628812:/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/c/Windows/System32:/c/Windows:/c/Windows/System32/wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files/nodejs:/c/ProgramData/chocolatey/bin:/c/Program Files/MATLAB/R2023b/bin:/c/Users/psych/anaconda3:/c/Users/psych/anaconda3/Scripts:/c/Users/psych/anaconda3/Library/bin:/c/Program Files/Git/cmd:/c/Users/psych/AppData/Local/Programs/Python/Launcher:/c/Users/psych/AppData/Local/Microsoft/WindowsApps:/c/texlive/2022/bin/win32:/c/Users/psych/AppData/Local/Programs/Microsoft VS Code/bin:/c/Users/psych/AppData/Roaming/npm:/c/Users/psych/Downloads/hugo_0.115.3_windows-amd64:/d/Downloads/flutter_projects/src/flutter/bin:/c/Program Files/RStudio/resources/app/bin/quarto/bin:/c/Program Files/RStudio/resources/app/bin/postback:/c/Users/psych/AppData:/c/Program Files/RStudio/resources/app/bin/quarto/bin:/c/Program Files/RStudio/resources/app/bin/postback/)
I whisper.cpp build info: 
I UNAME_S:  MSYS_NT-10.0-22631
I UNAME_P:  unknown
I UNAME_M:  x86_64
I PKG_CFLAGS:   -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600
I PKG_CPPFLAGS: -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600
I PKG_LIBS:  

gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
whisper_cpp/ggml-quants.c:1337:14: warning: 'make_qkx1_quants' defined but not used [-Wunused-function]
 1337 | static float make_qkx1_quants(int n, int nmax, const float * restrict x, uint8_t * restrict L, float * restrict the_min,
      |              ^~~~~~~~~~~~~~~~
gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
whisper_cpp/ggml-backend.c:841:13: warning: 'sched_print_assignments' defined but not used [-Wunused-function]
  841 | static void sched_print_assignments(ggml_backend_sched_t sched, struct ggml_cgraph * graph) {
      |             ^~~~~~~~~~~~~~~~~~~~~~~
gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:17593:13: warning: 'ggml_opt_get_grad' defined but not used [-Wunused-function]
17593 | static void ggml_opt_get_grad(int np, struct ggml_tensor * const ps[], float * g) {
      |             ^~~~~~~~~~~~~~~~~
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
whisper_cpp/whisper.cpp:203:29: warning: 'ggml_tensor* ggml_mul_mat_pad(ggml_context*, ggml_tensor*, ggml_tensor*, int)' defined but not used [-Wunused-function]
  203 | static struct ggml_tensor * ggml_mul_mat_pad(struct ggml_context * ctx, struct ggml_tensor * x, struct ggml_tensor * y, int pad = 32) {
      |                             ^~~~~~~~~~~~~~~~
In file included from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/hashtable_policy.h:36,
                 from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/hashtable.h:35,
                 from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/unordered_map.h:33,
                 from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/unordered_map:41,
                 from C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include/Rcpp/platform/compiler.h:153,
                 from C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include/Rcpp/r/headers.h:66,
                 from C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include/RcppCommon.h:30,
                 from C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include/Rcpp.h:27,
                 from whisper_cpp/whisper.cpp:1:
In static member function 'static void std::__copy_move<false, false, std::random_access_iterator_tag>::__assign_one(_Tp*, _Up*) [with _Tp = const whisper_grammar_element*; _Up = const whisper_grammar_element* const]',
    inlined from 'static _Up* std::__copy_move<_IsMove, true, std::random_access_iterator_tag>::__copy_m(_Tp*, _Tp*, _Up*) [with _Tp = const whisper_grammar_element* const; _Up = const whisper_grammar_element*; bool _IsMove = false]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:440:20,
    inlined from '_OI std::__copy_move_a2(_II, _II, _OI) [with bool _IsMove = false; _II = const whisper_grammar_element* const*; _OI = const whisper_grammar_element**]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:506:30,
    inlined from '_OI std::__copy_move_a1(_II, _II, _OI) [with bool _IsMove = false; _II = const whisper_grammar_element* const*; _OI = const whisper_grammar_element**]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:533:42,
    inlined from '_OI std::__copy_move_a(_II, _II, _OI) [with bool _IsMove = false; _II = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _OI = const whisper_grammar_element**]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:540:31,
    inlined from '_OI std::copy(_II, _II, _OI) [with _II = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _OI = const whisper_grammar_element**]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:633:7,
    inlined from 'static _ForwardIterator std::__uninitialized_copy<true>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, std::vector<const whisper_grammar_element*> >; _ForwardIterator = const whisper_grammar_element**]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_uninitialized.h:147:27,
    inlined from '_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _ForwardIterator = const whisper_grammar_element**]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_uninitialized.h:185:15,
    inlined from '_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, allocator<_Tp>&) [with _InputIterator = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _ForwardIterator = const whisper_grammar_element**; _Tp = const whisper_grammar_element*]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_uninitialized.h:373:37,
    inlined from 'std::vector<_Tp, _Alloc>::vector(const std::vector<_Tp, _Alloc>&) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:603:31,
    inlined from 'void std::__new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/new_allocator.h:187:4,
    inlined from 'static void std::allocator_traits<std::allocator<_Tp1> >::construct(allocator_type&, _Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:537:17,
    inlined from 'void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::vector<const whisper_grammar_element*>; _Alloc = std::allocator<std::vector<const whisper_grammar_element*> >]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:1283:30,
    inlined from 'void whisper_grammar_advance_stack(const std::vector<std::vector<whisper_grammar_element> >&, const std::vector<const whisper_grammar_element*>&, std::vector<std::vector<const whisper_grammar_element*> >&)' at whisper_cpp/whisper.cpp:4031:29:
C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:398:17: warning: array subscript 0 is outside array bounds of 'const whisper_grammar_element* [0]' [-Warray-bounds=]
  398 |         { *__to = *__from; }
      |           ~~~~~~^~~~~~~~~
In file included from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/x86_64-w64-mingw32.static.posix/bits/c++allocator.h:33,
                 from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/allocator.h:46,
                 from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:39,
                 from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/ext/alloc_traits.h:34,
                 from C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/hashtable_policy.h:39:
In member function '_Tp* std::__new_allocator<_Tp>::allocate(size_type, const void*) [with _Tp = const whisper_grammar_element*]',
    inlined from 'static _Tp* std::allocator_traits<std::allocator<_Tp1> >::allocate(allocator_type&, size_type) [with _Tp = const whisper_grammar_element*]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:482:28,
    inlined from 'std::_Vector_base<_Tp, _Alloc>::pointer std::_Vector_base<_Tp, _Alloc>::_M_allocate(std::size_t) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:378:33,
    inlined from 'std::_Vector_base<_Tp, _Alloc>::pointer std::_Vector_base<_Tp, _Alloc>::_M_allocate(std::size_t) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:375:7,
    inlined from 'void std::_Vector_base<_Tp, _Alloc>::_M_create_storage(std::size_t) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:395:44,
    inlined from 'std::_Vector_base<_Tp, _Alloc>::_Vector_base(std::size_t, const allocator_type&) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:332:26,
    inlined from 'std::vector<_Tp, _Alloc>::vector(const std::vector<_Tp, _Alloc>&) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:600:61,
    inlined from 'void std::__new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/new_allocator.h:187:4,
    inlined from 'static void std::allocator_traits<std::allocator<_Tp1> >::construct(allocator_type&, _Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:537:17,
    inlined from 'void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::vector<const whisper_grammar_element*>; _Alloc = std::allocator<std::vector<const whisper_grammar_element*> >]' at C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:1283:30,
    inlined from 'void whisper_grammar_advance_stack(const std::vector<std::vector<whisper_grammar_element> >&, const std::vector<const whisper_grammar_element*>&, std::vector<std::vector<const whisper_grammar_element*> >&)' at whisper_cpp/whisper.cpp:4031:29:
C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/new_allocator.h:147:55: note: object of size 0 allocated by 'operator new'
  147 |         return static_cast<_Tp*>(_GLIBCXX_OPERATOR_NEW(__n * sizeof(_Tp)));
      |                                                       ^
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c rcpp_whisper.cpp -o rcpp_whisper.o
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/psych/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c RcppExports.cpp -o RcppExports.o
g++ -shared -s -static-libgcc -o audio.whisper.dll tmp.def whisper_cpp/ggml-quants.o whisper_cpp/ggml-backend.o whisper_cpp/ggml-alloc.o whisper_cpp/ggml.o whisper_cpp/whisper.o whisper_cpp/common-ggml.o whisper_cpp/common.o rcpp_whisper.o RcppExports.o -LC:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/RBuildTools/4.4/x86_64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R/R-44~1.1/bin/x64 -lR
installing to C:/Users/psych/AppData/Local/R/win-library/4.4/00LOCK-audio.whisper/00new/audio.whisper/libs/x64
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (audio.whisper)

@jwijffels
Copy link
Contributor

You need to check if your hardware support these compilation flags: -mavx -mavx2 -mavx512f -mfma -mf16c -msse3 -mssse3

@chaoliu-cl
Copy link
Author

I am using Intel Core i7-1165G7 machine and my CPU supports the following relevant flags:

  • AVX2
  • AVX
  • FMA
  • SSE4.2
  • SSE4.1
  • SSSE3
  • SSE3
  • SSE2
  • SSE
  • MMX

For the package installation, I think the relevant flags are:

  • AVX
  • AVX2
  • FMA
  • F16C

Then I followed your code in the README:

Sys.setenv(WHISPER_CFLAGS = "-mavx -mavx2 -mfma -mf16c")
remotes::install_github("bnosac/audio.whisper", ref = "0.3.3", force = TRUE)
Sys.unsetenv("WHISPER_CFLAGS")

But I still had the abortion issue.

@jwijffels
Copy link
Contributor

What crash exactly are you getting?
Could you check from which cflag this is coming?
I can not reproduce this.

==> Rcpp::compileAttributes()

* Updated src/RcppExports.cpp
* Updated R/RcppExports.R

==> Rcmd.exe INSTALL --preclean --no-multiarch --with-keep.source audio.whisper

* installing to library 'C:/Users/jwijf/AppData/Local/R/win-library/4.4'
* installing *source* package 'audio.whisper' ...
** using staged installation
** libs
which: no nvcc in (/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/PROGRA~1/R/R-44~1.1/bin/x64:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/Program Files/R/R-4.4.1/bin/x64:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0:/c/windows/System32/OpenSSH:/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/c/Program Files/Positron/bin:/c/Program Files/PuTTY:/c/Program Files/Docker/Docker/resources/bin:/c/Program Files (x86)/Pulse Secure/VC142.CRT/X64:/c/Program Files (x86)/Pulse Secure/VC142.CRT/X86:/c/Program Files (x86)/Common Files/Pulse Secure/TNC Client Plugin:/c/Program Files/Git/cmd:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/150/Tools/Binn:/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Users/jwijf/AppData/Local/Programs/Python/Python312/Scripts:/c/Users/jwijf/AppData/Local/Programs/Python/Python312:/c/Users/jwijf/AppData/Local/Programs/Python/Launcher:/c/Users/jwijf/AppData/Local/Programs/Quarto/bin:/c/Users/jwijf/AppData/Local/Microsoft/WindowsApps:/c/Users/jwijf/AppData/Local/Programs/Microsoft VS Code/bin:/c/Users/jwijf/.dotnet/tools:/c/Users/jwijf/AppData/Local/Programs/Quarto/bin:/c/Program Files/RStudio/resources/app/bin/postback/)
using C++ compiler: 'G__~1.EXE (GCC) 13.2.0'
using C++11
which: no nvcc in (/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/PROGRA~1/R/R-44~1.1/bin/x64:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/Program Files/R/R-4.4.1/bin/x64:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0:/c/windows/System32/OpenSSH:/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/c/Program Files/Positron/bin:/c/Program Files/PuTTY:/c/Program Files/Docker/Docker/resources/bin:/c/Program Files (x86)/Pulse Secure/VC142.CRT/X64:/c/Program Files (x86)/Pulse Secure/VC142.CRT/X86:/c/Program Files (x86)/Common Files/Pulse Secure/TNC Client Plugin:/c/Program Files/Git/cmd:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/150/Tools/Binn:/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Users/jwijf/AppData/Local/Programs/Python/Python312/Scripts:/c/Users/jwijf/AppData/Local/Programs/Python/Python312:/c/Users/jwijf/AppData/Local/Programs/Python/Launcher:/c/Users/jwijf/AppData/Local/Programs/Quarto/bin:/c/Users/jwijf/AppData/Local/Microsoft/WindowsApps:/c/Users/jwijf/AppData/Local/Programs/Microsoft VS Code/bin:/c/Users/jwijf/.dotnet/tools:/c/Users/jwijf/AppData/Local/Programs/Quarto/bin:/c/Program Files/RStudio/resources/app/bin/postback/)
I whisper.cpp build info: 
I UNAME_S:  MSYS_NT-10.0-22621
I UNAME_P:  unknown
I UNAME_M:  x86_64
I PKG_CFLAGS:   -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600
I PKG_CPPFLAGS: -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600
I PKG_LIBS:  

which: no nvcc in (/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/PROGRA~1/R/R-44~1.1/bin/x64:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/x86_64-w64-mingw32.static.posix/bin:/usr/bin:/c/Program Files/R/R-4.4.1/bin/x64:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0:/c/windows/System32/OpenSSH:/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/c/Program Files/Positron/bin:/c/Program Files/PuTTY:/c/Program Files/Docker/Docker/resources/bin:/c/Program Files (x86)/Pulse Secure/VC142.CRT/X64:/c/Program Files (x86)/Pulse Secure/VC142.CRT/X86:/c/Program Files (x86)/Common Files/Pulse Secure/TNC Client Plugin:/c/Program Files/Git/cmd:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/150/Tools/Binn:/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Users/jwijf/AppData/Local/Programs/Python/Python312/Scripts:/c/Users/jwijf/AppData/Local/Programs/Python/Python312:/c/Users/jwijf/AppData/Local/Programs/Python/Launcher:/c/Users/jwijf/AppData/Local/Programs/Quarto/bin:/c/Users/jwijf/AppData/Local/Microsoft/WindowsApps:/c/Users/jwijf/AppData/Local/Programs/Microsoft VS Code/bin:/c/Users/jwijf/.dotnet/tools:/c/Users/jwijf/AppData/Local/Programs/Quarto/bin:/c/Program Files/RStudio/resources/app/bin/postback/)
I whisper.cpp build info: 
I UNAME_S:  MSYS_NT-10.0-22621
I UNAME_P:  unknown
I UNAME_M:  x86_64
I PKG_CFLAGS:   -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600
I PKG_CPPFLAGS: -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600
I PKG_LIBS:  

gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
whisper_cpp/ggml-quants.c:1337:14: warning: 'make_qkx1_quants' defined but not used [-Wunused-function]
 1337 | static float make_qkx1_quants(int n, int nmax, const float * restrict x, uint8_t * restrict L, float * restrict the_min,
      |              ^~~~~~~~~~~~~~~~
gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
whisper_cpp/ggml-backend.c:841:13: warning: 'sched_print_assignments' defined but not used [-Wunused-function]
  841 | static void sched_print_assignments(ggml_backend_sched_t sched, struct ggml_cgraph * graph) {
      |             ^~~~~~~~~~~~~~~~~~~~~~~
gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
gcc  -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"  -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -D_XOPEN_SOURCE=600   -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:17593:13: warning: 'ggml_opt_get_grad' defined but not used [-Wunused-function]
17593 | static void ggml_opt_get_grad(int np, struct ggml_tensor * const ps[], float * g) {
      |             ^~~~~~~~~~~~~~~~~
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
whisper_cpp/whisper.cpp:203:29: warning: 'ggml_tensor* ggml_mul_mat_pad(ggml_context*, ggml_tensor*, ggml_tensor*, int)' defined but not used [-Wunused-function]
  203 | static struct ggml_tensor * ggml_mul_mat_pad(struct ggml_context * ctx, struct ggml_tensor * x, struct ggml_tensor * y, int pad = 32) {
      |                             ^~~~~~~~~~~~~~~~
In file included from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/hashtable_policy.h:36,
                 from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/hashtable.h:35,
                 from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/unordered_map.h:33,
                 from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/unordered_map:41,
                 from C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include/Rcpp/platform/compiler.h:153,
                 from C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include/Rcpp/r/headers.h:66,
                 from C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include/RcppCommon.h:30,
                 from C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include/Rcpp.h:27,
                 from whisper_cpp/whisper.cpp:1:
In static member function 'static void std::__copy_move<false, false, std::random_access_iterator_tag>::__assign_one(_Tp*, _Up*) [with _Tp = const whisper_grammar_element*; _Up = const whisper_grammar_element* const]',
    inlined from 'static _Up* std::__copy_move<_IsMove, true, std::random_access_iterator_tag>::__copy_m(_Tp*, _Tp*, _Up*) [with _Tp = const whisper_grammar_element* const; _Up = const whisper_grammar_element*; bool _IsMove = false]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:440:20,
    inlined from '_OI std::__copy_move_a2(_II, _II, _OI) [with bool _IsMove = false; _II = const whisper_grammar_element* const*; _OI = const whisper_grammar_element**]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:506:30,
    inlined from '_OI std::__copy_move_a1(_II, _II, _OI) [with bool _IsMove = false; _II = const whisper_grammar_element* const*; _OI = const whisper_grammar_element**]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:533:42,
    inlined from '_OI std::__copy_move_a(_II, _II, _OI) [with bool _IsMove = false; _II = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _OI = const whisper_grammar_element**]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:540:31,
    inlined from '_OI std::copy(_II, _II, _OI) [with _II = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _OI = const whisper_grammar_element**]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:633:7,
    inlined from 'static _ForwardIterator std::__uninitialized_copy<true>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, std::vector<const whisper_grammar_element*> >; _ForwardIterator = const whisper_grammar_element**]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_uninitialized.h:147:27,
    inlined from '_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _ForwardIterator = const whisper_grammar_element**]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_uninitialized.h:185:15,
    inlined from '_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, allocator<_Tp>&) [with _InputIterator = __gnu_cxx::__normal_iterator<const whisper_grammar_element* const*, vector<const whisper_grammar_element*> >; _ForwardIterator = const whisper_grammar_element**; _Tp = const whisper_grammar_element*]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_uninitialized.h:373:37,
    inlined from 'std::vector<_Tp, _Alloc>::vector(const std::vector<_Tp, _Alloc>&) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:603:31,
    inlined from 'void std::__new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/new_allocator.h:187:4,
    inlined from 'static void std::allocator_traits<std::allocator<_Tp1> >::construct(allocator_type&, _Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:537:17,
    inlined from 'void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::vector<const whisper_grammar_element*>; _Alloc = std::allocator<std::vector<const whisper_grammar_element*> >]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:1283:30,
    inlined from 'void whisper_grammar_advance_stack(const std::vector<std::vector<whisper_grammar_element> >&, const std::vector<const whisper_grammar_element*>&, std::vector<std::vector<const whisper_grammar_element*> >&)' at whisper_cpp/whisper.cpp:4031:29:
C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_algobase.h:398:17: warning: array subscript 0 is outside array bounds of 'const whisper_grammar_element* [0]' [-Warray-bounds=]
  398 |         { *__to = *__from; }
      |           ~~~~~~^~~~~~~~~
In file included from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/x86_64-w64-mingw32.static.posix/bits/c++allocator.h:33,
                 from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/allocator.h:46,
                 from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:39,
                 from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/ext/alloc_traits.h:34,
                 from C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/hashtable_policy.h:39:
In member function '_Tp* std::__new_allocator<_Tp>::allocate(size_type, const void*) [with _Tp = const whisper_grammar_element*]',
    inlined from 'static _Tp* std::allocator_traits<std::allocator<_Tp1> >::allocate(allocator_type&, size_type) [with _Tp = const whisper_grammar_element*]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:482:28,
    inlined from 'std::_Vector_base<_Tp, _Alloc>::pointer std::_Vector_base<_Tp, _Alloc>::_M_allocate(std::size_t) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:378:33,
    inlined from 'std::_Vector_base<_Tp, _Alloc>::pointer std::_Vector_base<_Tp, _Alloc>::_M_allocate(std::size_t) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:375:7,
    inlined from 'void std::_Vector_base<_Tp, _Alloc>::_M_create_storage(std::size_t) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:395:44,
    inlined from 'std::_Vector_base<_Tp, _Alloc>::_Vector_base(std::size_t, const allocator_type&) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:332:26,
    inlined from 'std::vector<_Tp, _Alloc>::vector(const std::vector<_Tp, _Alloc>&) [with _Tp = const whisper_grammar_element*; _Alloc = std::allocator<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:600:61,
    inlined from 'void std::__new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/new_allocator.h:187:4,
    inlined from 'static void std::allocator_traits<std::allocator<_Tp1> >::construct(allocator_type&, _Up*, _Args&& ...) [with _Up = std::vector<const whisper_grammar_element*>; _Args = {const std::vector<const whisper_grammar_element*, std::allocator<const whisper_grammar_element*> >&}; _Tp = std::vector<const whisper_grammar_element*>]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/alloc_traits.h:537:17,
    inlined from 'void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = std::vector<const whisper_grammar_element*>; _Alloc = std::allocator<std::vector<const whisper_grammar_element*> >]' at C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/stl_vector.h:1283:30,
    inlined from 'void whisper_grammar_advance_stack(const std::vector<std::vector<whisper_grammar_element> >&, const std::vector<const whisper_grammar_element*>&, std::vector<std::vector<const whisper_grammar_element*> >&)' at whisper_cpp/whisper.cpp:4031:29:
C:/rtools44/x86_64-w64-mingw32.static.posix/lib/gcc/x86_64-w64-mingw32.static.posix/13.2.0/include/c++/bits/new_allocator.h:147:55: note: object of size 0 allocated by 'operator new'
  147 |         return static_cast<_Tp*>(_GLIBCXX_OPERATOR_NEW(__n * sizeof(_Tp)));
      |                                                       ^
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c rcpp_whisper.cpp -o rcpp_whisper.o
g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-44~1.1/include" -DNDEBUG -O3 -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -I'C:/Users/jwijf/AppData/Local/R/win-library/4.4/Rcpp/include'   -I"C:/rtools44/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign  -c RcppExports.cpp -o RcppExports.o
g++ -shared -s -static-libgcc -o audio.whisper.dll tmp.def whisper_cpp/ggml-quants.o whisper_cpp/ggml-backend.o whisper_cpp/ggml-alloc.o whisper_cpp/ggml.o whisper_cpp/whisper.o whisper_cpp/common-ggml.o whisper_cpp/common.o rcpp_whisper.o RcppExports.o -LC:/rtools44/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools44/x86_64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R/R-44~1.1/bin/x64 -lR
installing to C:/Users/jwijf/AppData/Local/R/win-library/4.4/00LOCK-audio.whisper/00new/audio.whisper/libs/x64
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (audio.whisper)

> library(audio.whisper)
> model <- whisper("tiny")
trying URL 'https://huggingface.co/ggerganov/whisper.cpp/resolve/d15393806e24a74f60827e23e986f0c10750b358/ggml-tiny.bin'
Content type 'application/octet-stream' length 77691713 bytes (74.1 MB)
downloaded 74.1 MB

Downloading finished, model stored at 'C:/Users/jwijf/Desktop/open-source/audio.whisper/ggml-tiny.bin'
whisper_init_from_file_with_params_no_state: loading model from 'C:/Users/jwijf/Desktop/open-source/audio.whisper/ggml-tiny.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 384
whisper_model_load: n_audio_head  = 6
whisper_model_load: n_audio_layer = 4
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 384
whisper_model_load: n_text_head   = 6
whisper_model_load: n_text_layer  = 4
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 1 (tiny)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:      CPU buffer size =    77.18 MB
whisper_model_load: model size    =   77.11 MB
whisper_init_state: kv self size  =    8.26 MB
whisper_init_state: kv cross size =    9.22 MB
whisper_init_state: compute buffer (conv)   =   12.17 MB
whisper_init_state: compute buffer (encode) =   64.92 MB
whisper_init_state: compute buffer (cross)  =    4.01 MB
whisper_init_state: compute buffer (decode) =   96.02 MB
> audio <- system.file(package = "audio.whisper", "samples", "jfk.wav")
> trans <- predict(model, newdata = audio, language = "en", n_threads = 2)
system_info: n_threads = 2 / 32 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0 | 
Processing C:/Users/jwijf/AppData/Local/R/win-library/4.4/audio.whisper/samples/jfk.wav (176000 samples, 11 sec), lang = en, translate = 0, timestamps = 0, beam_size = -1, best_of = 5
[00:00:00.000 --> 00:00:10.500]   And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.

@jwijffels
Copy link
Contributor

Maybe try with R 4.1.3 and RTools 4.0

@chaoliu-cl
Copy link
Author

chaoliu-cl commented Oct 4, 2024

I tested on some other devices. The other Intel machine (i5-11500T) also had a crash (R aborted without any error messages) but the AMD machine (Ryzen 7 PRO 7730U) was fine. They should all support these flags: AVX, AVX2, FMA, F16C. Below see the pictures for more information on these two devices:
image

image

@jwijffels
Copy link
Contributor

You might want to set WHISPER_CFLAGS to "" when installing the package and gradually add more flags to see what is the culprit.
Or it might as well be linked to the compiler you have

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants