Google Summer of Code 2022 Final report

Title - Music Blocks V3 Maintenance

Organization - Sugarlabs

This is a elucidation of my work on V3 Maintenance during Google Summer of Code 2022 with Sugar Labs. This repository contains links, samples of my work, and examples.The code samples for changes related to improvements and bug fixes are present in authored files.

📙 Abstract

Music Blocks is a Visual Programming Language and collection of manipulative tools for exploring musical and mathematical concepts in an integrative and fun way.

The purpose of my project is to maintain and fix

• Loading of requirements {Primary}
  1. Avoid race conditions
  2. Ensure Android Version is reasonably robust
  3. To ensure better and reliable loading experience in both chrome and firefox.
  4. Research on how to manage loading requirements for large projects
• Planet’s cross origin errors
• Regression in music utils code
• Better characterization of code
• Other bugs in v3 which can be resolved
• Documentation Maintenance

📝 Tech Stack

I've used my intensive experience with HTML5, CSS and JavaScript to fix and improve many features of version-3 since these are the primary languages used to build musicblocks V3.

HTML5 includes detailed processing models to encourage more interoperable implementations; it extends, improves, and rationalizes the markup available for documents and introduces markup and application programming interfaces (APIs) for complex web applications.

CSS is designed to enable the separation of presentation and content, including layout, colors, and fonts. This separation can improve content accessibility; provide more flexibility and control in the specification of presentation characteristics; enable multiple web pages to share formatting by specifying the relevant CSS in a separate .css file, which reduces complexity and repetition in the structural content; and enable the .css file to be cached to improve the page load speed between the pages that share the file and its formatting.

JavaScript is a high-level, often just-in-time compiled language that conforms to the ECMAScript standard. It has dynamic typing, prototype-based object-orientation, and first-class functions. It is multi-paradigm, supporting event-driven, functional, and imperative programming styles. It has application programming interfaces (APIs) for working with text, dates, regular expressions, standard data structures, and the Document Object Model (DOM).

In addition, for maintaining the performance of the application, many latest tools are used that help in website's optimizations by providing insights on various metrics and suggest opportunities to improve by diagnosing performance issues:-

In the beginning, I explored many web application performance metrics website to check loading reliability and performance:-

Performance and availability monitoring
Pagespeed Insights (Google)
GT Metrix (Carbon 60)
Pingdom (SolarWinds)

Finally, I decided to go with PageSpeed Insights. I have chosen this platform because it can evaluate all sorts of factors, including performance, accessibility, progressive web apps, and more. Apart from that, PageSpeed Insights (PSI) reports on the performance of a page on both mobile and desktop devices, and provides suggestions on how that page may be improved. PSI provides both lab and field data about a page. Lab data is useful for debugging performance issues, as it is collected in a controlled environment .

Also, I have used FFmpeg to convert GIF to .mp4 and .webm to improve loading speed and reliability.

FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. It supports the most obscure ancient formats up to the cutting edge. No matter if they were designed by some standards committee, the community or a corporation. It is also highly portable: FFmpeg compiles, runs, and passes our testing infrastructure FATE across Linux, Mac OS X, Microsoft Windows, the BSDs, Solaris, etc. under a wide variety of build environments, machine architectures, and configurations.

✨ Work Progression

In the community bonding period, the work was mostly confined to coming up with an updated time-line with the detailed list of tasks and their potential solutions. In this period there were a lot of meetings and discussions with mentors on video and text based platforms and through extensive communication a timeline was drawn that was in the best interest of the project. This period was also used for introductions and getting to know fellow contributors and also getting used to and familiar with the organization’s work culture and best practices.

Key-events during Community Bonding Period

• Creation of updated timeline - exhaustive list of objectives, a narrative of the features to build, and a list of implementation detail segmented in 12 equal parts
  • Gsoc Timeline
• Getting to know the codebase of music blocks
• Communication with mentors regarding the details of the project

My work, basically, consisted of four major parts:

1. Loading speed and reliability of web application
2. Resolving Cross Origin Problems
3. Regression in music utils code
4. Documentation Maintenance

Since, major part of the code cannot be changed as version-4 is in the making for better characterization and modularity, so the maintenance is majorly focused on small changes that can leave maximum impact on performance of the application.

📝 Loading of Requirements

Musicblocks v3 is a large project with a lot of modules and functionality, and hence it is very important to follow the best practices when managing it's loading requirements. Initially, the speed and relaibility of v3 requirements loading was very slow on both chrome and firefox.

The report by PageSpeed Insights :-

Overall Performance Score :- 29/100

Resolution :-

1. Deleted -- /loading-animation.gif size- 3.44mb (3612912 kb) -> Added -- /loading-animation.mp4 size - less than 260kb

   Estimated Savings - 2.66 seconds

   Animated GIFs have three key traits that a video needs to replicate:

   • They play automatically.
   • They loop continuously (usually, but it is possible to prevent looping).
   • They're silent.
   Using video tag and it's attributes recreated these behaviors.

   Details on how to convert artwork[GIF] to video is elaborated here.

   This step solves many problems of loading issue:-

   • Use video formats for animated content with proper sizing. (Savings - 2.66s )
   • Largest contentful paint. (40s to less than 2s, savings - more than 30s)
   • Makes the android version robust.
2. Changing img to amp-img

   AMP may choose to delay or prioritize resource loading based on the viewport position, system resources, connection bandwidth, or other factors. The amp-img components allows the runtime to effectively manage image resources this way.

   Potential savings :- Memory = 276.5kb + 18.3kb Time = 0.2s

Eliminate render blocking resources was also a potential method to save more load time however these changes make the index.html file very unreadable and have a little or no affect on loading reliability of requirements and thus making a script inline is not a very efficient solution to the problem at hand , henceforth will be discontinued . Learn More.

After implementation:-

Overall Performance Score :- 76/100

Loading reliability and speed is increased from 29/100 to 76/100 . Improvement of more than 162% .

This issue addresses the barebones and major sections by which loading reliability is very low:

🐛 Issue	#3035 #3039 #3042 #3045 #3046 #3047 #3048	Loading Of Requirements
🔀 PR	#3043 #3048	Loading Of Requirements PR
🎉 Commits	6	all-commits

There were total of six commits consisting of defer offscreen images, using video formats and proper sizing of media files.

📝 Cross Origin Errors

Before:-

Solution:-

Cross origin is arising due to the fonts that are being accessed from google API , removing that section of code resolves this issue.

After:-

🐛 Issue	#3049	Cross origin errors
🔀 PR	#3050	cross origin errors removed
🎉 Commits	1	all-commits

📝 Regression in music utils

For the blocks :-

F♯4, G♯4, A♯4, B4, C♯5, D♯5, E♯5 F♯5, G♯5, A♯5, B5, C♯6, D♯6, E♯6 is what one would expect to see So the transition from D#5 to F5 is where things begin to break down. The F# through A# are skipped for some reason. Maybe it gets confused when it does the conversion from D# to F (which it really should not be doing in F# Major)

Solution:-

F# major is broken due to a nomenclature anomaly F# major (full, full, half, full, full, full, half) is: F# -> G# -> A# -> B -> C# -> D# -> E# -> (+1)F# this is the internal representation used F#, G#, A#, B, C#, D#, E#, F# the thing is ... E# is same as F so the code is trying to find F in the scale (list) but the list instead has E# improve on the "find in list". Instead of string equality if you can find logical equality it should fix it. Say something like obj.findIndex(item => logicalEquals(item, thisPitch)) the new logicalEquals function should return true for E# and F, E and Fb, etc but false for say E# and F# whenever there are two white keys in a row. E==Fb and E#==F; B==Cb and B#==C

After implementation:-

🐛 Issue	#3055	Regression in musicutils code
🔀 PR	#3054	Regression Resolved
🎉 Commits	3	all-commits

There were total three commits consisting of actual code change and then formatting of code through prettier to follow the style of rest of the code.

📝 Documentation Maintenance

This work was basically divided into two major portions, first to fix the way image is linked to readme files as the link to images(svg files) should appear in proper format showing the hierarchy of folders and files where it is located instead of camo image link that was appearing before and added missing image example like no-clock.svg .

Second, was set-key documenation, in which the proper description was to be added for set-key block with an example and table of how key, modes and pattern changes with the value of set key block.

Set Key

🐛 Issue	#3061	Image link in docx
🔀 PR	#3069	fixed img link and added no-clock.svg
🔀 PR	#3070	Set-Key
🎉 Commits	15	set key and img link commits

There were a total of *fifteen* commits which include adding image, table and description of set key and image link fix with no-clock.svg.

🚀 Other bugs and fixes

Other than the main aim of the project, along the way I also made fixes that will ease the work for future contributors , and also become very handy to the users of v3.

• Deleted galton-music.html (unnecessary file)
• Removed Warning when longer than 2 notes are played.

🐛 Issue	#3065	Warning for long notes
🔀 PR	#3067	Removed warning
🔀 PR	#3041	Deleted galton-music.html

✨ Enhancements

There are many enhancements that can be done particularly to improve characterization and modularity of code which will further improve the loading reliability, also there are many small bugs which can be resolved which are listed in the issues section of the repository.

Some issues that are particularly fun to solve are :-

1. Remove Arrow

2. Wav files

3. Drag in touchscreen

👨 Acknowledgements

On a final note, I am extremely grateful to my mentors, Walter Bender, Devin Ulibarri and Anindya Kundu.I am also very thankful for their regular feedbacks and constant support throughout which helped me in improving the quality of my code and helping me improve my soft skills which reulted in successful completetion of the project within the time period.

Thanks to Google, GSoc ,Sugarlabs and MusicBlocks for this great opportunity.

Thanks

Ashutosh Gautam