Planning a new, modern and stable NewPipe

[Stypox]

Hey guys. You might have noticed the somewhat slower and haphazard updates to NewPipe for the last few releases. The thing is, NewPipe was started, like so many others in FOSS, as an enthusiast project. It's been over 7 years since then, and the age of the codebase is starting to show through spaghetti code, outdated UI, bad design choices and instability.

It has gotten to the point that we're having difficulties even maintaining the current code and fixing bugs, let alone adding new features, due to complex interconnections and lack of proper modularity. Progress has been slow, since developers were focused on fixing nasty bugs. Often users requested features that would make sense, but nobody had the time (and, sometimes, knowledge of the codebase) to implement them.

To mitigate the issues we initially tried to do incremental refactors of various components. Take for example the player, which has often been deemed "a hot pile of garbage": it underwent a couple major Pull Requests that restructured it while maintaining all features. Although now the code is a bit better, new crashes and unwanted behaviors have arisen, because the interaction between the old and the new code is not optimal. We tried again and again to debug the problems, without luck.

At the moment NewPipe is relying on libraries and components that have become outdated, which might not only drive users away from the app, but also push away developers wanting to get some experience on new technologies. We need to migrate to Material Design 3, Kotlin and Jetpack Compose, and unfortunately doing so basically requires rewriting everything.

So, we've discussed and discussed this ad nauseam. We're very sure the old code needs to go, no matter what, to be replaced by a modern and professional codebase. But we can't decide on the approach to get there.

Anyone who's been in the FOSS community for long has come across at least one or two failed rewrites or refactors, and we don't want to repeat such tragic history. We believe NewPipe really has the potential to be a staple app in people's devices, if only we could get across this massive hurdle and come out the other side relatively intact.

What we need from you guys is advice. Preferably, experienced advice. Do we just dump the old code into maintenance mode and start afresh? Or replace it piece by piece like the Ship of Theseus? A combination of the two? Or something else entirely?

Please let us know. Also, if you don't have any directly helpful advice, please, please restrict yourself to just upvoting or leaving a thumbs up instead of commenting, so that there is less noise and more info in the thread. We all know how fantastic walls of text are at putting us off from reading, and we'd like to avoid that.

Also, if you are experienced in Android mobile development and would like to help us out with the rewrite, be sure to reach out to us! We will be using the most modern components whenever possible, as detailed below, so it is sure going to be a good learning experience for everyone involved. As a first step, please join the IRC from the ReadMe.

By the way, don't worry, all currently open PRs that introduce big features will be merged in NewPipe 0.26.0 or future releases of the current app. See the related Project and also take a look at the plans we have for the current app below.

Thanks! The NewPipe team.

What approach do we want to use?

• migrate the existing codebase of the app:
  • => risks:
    • strange issues between new and old code;
    • keeping an architecture which could be bad in some places;
    • going nowhere;
    • lack of interest from external contributors.
  • => benefits:
    • staying on one repository only;
    • better handling of issues.
• create a new app and reuse good code:
  • => risks:
    • a lot of time before reaching feature parity with the current app;
    • abandonment of devs due to the huge work required.
  • => benefits:
    • creating a modularized app in an easier way, and following good architecture and dependency models, hence making migration processes easier too;
    • potential increase of interest for NewPipe projects, which could lead to more (quality) contributions and to new team members;
    • embedding more community input in the app.

Similar or common issues are being raised increasingly in the Extractor repo, so these questions could be considered for the Extractor too.

Key elements we should use and not forget during development

• Good design thinking: possibility of contacting UI/UX designers and using community feedback to create new app designs:

  • => use Material 3 design system

  • => think about accessibility and form factors

• Asking feedback from the community and taking it into account where that's relevant: results of a subreddit poll returns around 80% in favour;

• Better organization: good use of Projects, which we would use to plan what should be in the next releases, what is in progress, what's remaining

  • => we've started to do this in the recent release, and we should grow our usage of these tools

• Try to get new and experienced people into the team: most of us are students passionate about technology, but:

  • we have other things to do in our life, and some or all of us may leave the NewPipe project at some point;

  • we can't know everything, and we lack knowledge in some domains.

• Apply more agile principles such as doing smaller but more frequent release cycles.

• For the development by itself:

  • Jetpack Compose;
  • ViewModels;
  • Hilt for dependency injection;
  • modules for app components (player, downloader, channel, search, settings, ...) with usage of interfaces for communication between them;
  • use Kotlin everywhere in the app to reduce code and bugs;
  • Jetpack Media3 for the player.
  • use https://penpot.app to create designs (it's an OSS alternative to Figma)

Decide what we want to ship in the current app

• highly requested features, mostly the ones which are blockers for content accessibility (also see the related Project):

  • comment replies (almost ready);
  • channel tabs (almost ready);

  => work of these features probably needs to be done by the team due to code and/or approaches from external contributions not quite meeting our standards

  • multiple audio tracks support (already merged, released in 0.25.2);
  • advanced search filters (could be a blocker for channel tabs due to the reworking of filters in the extractor);
  • better quality of images, by introducing multiple images support (a pull request was opened).

• bug fixes and regressions, in particular the most annoying ones (we can't guarantee we will be able to debug everything though, for the aforementioned reasons):

  • fullscreen issues on some devices since the player UI rewrite;
  • player "App Not Responding" and crashes on some devices and in some cases;
  • ViewHolders not attached crash;
  • fullscreen button with autorotation enabled.

[zedlabs]

[svenjacobs]

I just want to add my two cents:

If you're considering a (full) rewrite, you should really consider using Compose for all new UI. My colleagues and I have migrated an existing app in a professional context from XML to solely Compose within 1.5 years and we never regretted it! It's just mind-blowing how much you can achieve with so little code, especially when combining Compose with other recommended, modern architecture like ViewModels, Coroutines and UDF.

Compose is still relatively new so it's true that there is a learning curve, especially when you worked with views and XMLs for many, many years. The concepts are totally different but once you understand them, you will master Compose and don't want to go back 😉 There might still be a few quirks or things that worked with XML but don't work with Compose yet. However there are regular Compose updates and the Compose open-source community with great libraries also grows every day.

While Compose and Kotlin Multiplatform (Mobile) work well together, they are two separate frameworks. You can use one without the other. You could create a KMP/KMM app that uses XML for Android and UIKit for iOS, or Compose for Android and SwiftUI for iOS or Compose for every platform as there is Compose Multiplatform.

You see, there are many technological choices to be made. Whether you use KMM or not, you should really consider Compose for the Android part of the project. Not only is it the recommendation by Google, but it will also make development easier, faster and surely attract new developers 😃

[ticmaidev]

can you share an example, from what I've seen performance generally goes down when going from XML -> compose, as is noted by most professional android devs

this is real!

I've been working on an app for API level 25 designed for very poor hardware devices. While working with compose is way easier, it's very difficult to maintain good performance. Just search on YouTube about performance tips and instantly feel overwhelmed.

If you go with Jetpack Compose, I can already see NewPipe feed lists lagging a lot, even if you're using lazy loading.

Try installing any demo apps provided by Google on Jetpack repo, even their engineer apps lag a lot.

[BenjyTec]

@ticmaidev The lagging that can be seen in the demo apps probably comes from using the debug build variant. As soon as you build a release version of the app with minify etc, the performance should be drastically increased.
See also these posts:
https://medium.com/androiddevelopers/why-should-you-always-test-compose-performance-in-release-4168dd0f2c71
https://stackoverflow.com/a/70596559/8331157

[ticmaidev]

@BenjyTec no it's not. I've release compiled both demo and my own app. Scrolling with lazy grids, columns or rows are not as smooth as xml. But we could always "cheat" and use xml where we need more performance (mix compose and xml).

Btw, our previous app was completely made with Kotlin+XML and there's no lag. It's smooth af

[rajarshikhatua100]

I value performance over developer productivity for an Android app, but it can be opposite when it comes to meta's or amazon's large scale codebase (not Android apps)

[wardellbagby]

To be completely honest? It's a lose-lose. This is a tale absolutely as old as time and no matter which decision you go with, it's going to seem like everyone is unhappy.

I say this both as a fellow OSS maintainer and as an Android developer.

I know how easy it is to fall into the pitfall of "we'll write it GOOD this time!" but that almost never works out as well as we hope it does. The alternative approach of replacing your wheels while you're driving is harder to manage, though.

So my vote is for the big rewrite. Move the current NewPipe into legacy mode, shifting gears to pretty much only fix the worst bugs and get cracking on something new. At the very least, it'll be more rewarding to develop and you'll have an easy comparison metric. There's something to be said about the delight you feel loading up your old and new versions of the app against each other and seeing your new code work so much better than your old.

Plus, a more modern code could entice more contributors. Realistic, I know it almost never does but it's at least one less barrier to entry, right?

As a user, I think NewPipe is in a fine place right now. Yes, it has bugs but it's a completely free app that lets me access so many different services. Just keeping the current feature set is more than enough for me. Although of course I'd love to see a NewerPipe. 😂

I would also remind you that it's generally the most annoying users that are the loudest. I know you all have gotten a lot of flack for the amazing service you've provided but, as much as possible, try not to let the loudmouths influence your decision here.

I appreciate all of the difficult work that you all have done! Whatever the choice is, I hope you all are taking good care of yourself and your mental health! Burnout is serious!

[MeLlamoPablo]

I'm curious as to why do you vote for the big rewrite after identifying the lose-lose nature of the issue?

I agree that it is a lose-lose, but IMO it is a devs-lose/users-lose. The worst case scenario of each would be:

• App migration: the old code is never truly eliminated, devs lose interest, features and fixes are slower to ship.
  • If the migration is done via two entrypoints to the app, "the big switch" may never happen.
  • Devs lose, users remain roughly as happy as today.
• Big rewrite: NewerPipe never reaches feature parity, the userbase is scattered across the two, there's twice as many code to maintain.
  • Users lose, devs are happier if they work on NewerPipe or as happy as today if they work on NewPipe.

I'd argue that from those two scenarios we should choose the one in which users don't lose.

[wardellbagby]

If the developers lose, NewPipe ceases to work for everyone and dies.
If the users lose, NewPipe loses or splits users across multiple apps, but there's still a future where things get better again.

For me, it really comes down to the fact that this is an unpaid and open-source project. The developers owe us nothing so we should endeavor to keep them happy instead of the other way around.

[Maxr1998]

I wholeheartedly agree on the lose-lose situation part and would also vouch for the complete rewrite (but without knowing how bad the current code actually is).

I personally rewrote the official Android app for Jellyfin back in 2020, so I'll share my experience here as it may be insightful.

I started the rewrite immediately after joining the project, because the previous app used Cordova and was [insert less favorable reviews of the codebase] and the only real option was a rewrite. While I mostly consider the rewrite a success for both users and contributors, it's still far from perfect and only a few years later, there have been (and still are) many areas of regret in the code. Partially because some of the old code that was copied over or only slightly adapted is still trash (unstable, hard to understand, terrible to maintain and too complex to just rewrite, too), but also because the "we'll do it good this time" rarely works as expected:
I made a lot of bad decisions in the rewrite, and had to refactor some portions of the app multiple times to finally (may or may not actually be final) be happy with them. And especially when working with new technologies (like Compose), just getting into them doesn't help with writing clean code that follows best practices.
And lastly, I doubt a rewrite with new technologies is that impactful in recruiting new contributors. From my experience with Jellyfin Android (and I believe that even more applies to NewPipe), a modern tech stack doesn't help in all the highly specialized and complex business logic. Having appropriate documentation on those parts (in code and as contribution guide with a general overview) would help potential contributors help to quickly understand the structure and inner workings of the app so that they can move on to make their proposed changes. Thus, a rewrite should primarily be seen as a way to make the experience better for the current developers, and maybe also allow them to consider the documentation part right from the start.

That said, trying to reengineer an app on the existing codebase is much more complicated and a rewrite would offer some quick initial progress that might relatively quickly offer 60-80% of the use-cases and would allow early migration and feedback from some dedicated users who don't need all features without impacting those still on the legacy version.

[V3ntus]

Going off of the above comment's suggestion against Jetpack Compose, is there any interest in the consideration of Flutter?

As a big Flutter enthusiast myself, I would be interested in contributing heavily if that choice was made, but I do not have any knowledge of Jetpack or kotlin.

[opusforlife2]

Yattee already exists to make Apple users happy. We don't need to care about iOS/iPadOS because it is already 'handled'.

[gschurck]

Yattee already exists to make Apple users happy. We don't need to care about iOS/iPadOS because it is already 'handled'.

Yes indeed !
For a company it makes sense to build a project faster and easily for both platforms, but for an OSS project I think it's better to focus on the quality of the app.

1. Quality over quantity
2. Especially in Open Source, if your app is not good enough for the platform there will always be alternative that will be developed to better target this platform, if it doesn't exists already.

[FireMasterK]

While I think Flutter is a great project, I'm not sure if it's a great choice given the amount of interfacing you'll have to do with Java code for using NewPipeExtractor for use in Dart. (iOS doesn't support Java? Shipping a JRE on Linux/Windows/OS X would be a lot of bloat?)

I also feel a native Android application would perform better than a Flutter application, but that's just a personal opinion, without any actual actual concrete evidence to prove that.

There's also a lack of a good cross-platform media player for Flutter the last time I checked. media_kit however has almost bridged the gap and the situation has improved a lot since the last time I checked it.

I'm the author of Piped, a related project that uses NewPipeExtractor. Piped will not have the above mentioned issues as interfacing with a HTTP API is a lot easier than integrating NewPipeExtractor.

I have written a extremely WIP prototype that supports a trending and video playback screen I wrote many months back. My primary goals at that time were to support Linux, Windows, iOS and OS X (in that particular order).

If anyone is really interested in continuing and maintaining a maybe official Piped client in Flutter, please join the Piped room and let me know, and we can collaborate further. I can open-source my existing work if it helps. I don't want to derail this discussion.

[nospace-here]

On the other hand, it would be cool to have it be cross-platform (iOS might be getting sideloading capabilities at some point). But yeah it's up to the lead developer to make that decision.

Doesn't iOS development require mac-hardware and a costly developer license to develop them properly?
These costs come before even publishing the app anywhere and (current) Hackintoshes tend to be less stable than Windows.

[Stypox]

Well, Jetpack Compose is going multiplatform thanks to kotlin-multiplatform: https://www.jetbrains.com/lp/compose-multiplatform/
It is already available for iOS, desktop and web, so I bet fuchsia will join this list when it comes out. Also, this means that Jetpack Compose will allow us to possibly ship NewPipe to other platforms just like Flutter would.

[Sym8]

I'm just an end user of newpipe, if you guys wanna rewrite sure go ahead. You will have more freedom and better choices to make, currently there are alternative so users aren't left stranded. Better to rewrite now than to abandon the project 2 years later cause the code was clunky.

[dvdkon]

I vote for a rewrite. As a user, I'm really excited for a new development direction that could make Newpipe more stable and featureful. As a dev with some Jetpack Compose experience, I'm excited for a new maintainable, shinier codebase (even though I have too many side projects already...).

I'd actually advise against thinking about the new codebase's design too much. Overthinking can be as bad as "underthinking", and if the whole thing isn't a ball of spaghetti, it's never too late to rip out a component and start over. Not every future feature needs to be accomodated from the start.

Good luck :)

[gajwani]

I'm a web developer, with experience in Kotlin and big bang refactors, and I would say try for a middle ground between all-at-once and piecemeal. (We should) Create a plan for large chunks of the code that need refactoring and work on those large chunks (the player was a good example). This ought to help make progress, while still avoiding the perpetually incomplete rewrite and the splintering of the user base.

That all being said, I don't have too much Android experience, so I may not be considering the issues that arise as a result of Android-specific challenges.

[medavox]

This sounds reasonable - come up with a high-level plan for what needs fixing, and maybe tackle a module, a screen, or a feature at a time. Whatever is a reasonable chunk of work

The main thing to avoid is having too much unfinished work "in-flight" at the same time. The bigger it is, harder it is to finish, the longer it takes to review, and the more likely it will be perpetually outdated by faster PRs

[uragiristereo]

I have a decent experience developing with Jetpack Compose and I'm interested in to be a part of the rewrite of NewPipe project team.
I vote for a complete rewrite since using the modern tech stacks not only just updating a version, it also shifted the paradigm especially in the UI layer with Jetpack Compose.

[imashnake0]

Also down to contribute if we end up going this route. Admittedly, a lot of the m3/m2 components can have strange behaviour but Compose makes it easy to make composables of our own so this is not a big deal. @Stypox can we create a Discord server if we don't have one already?

[lm41]

I would also like to participate at the rewrite.

[YongJunLim]

Be it a migration or complete rewrite, I'm keen
on contributing to NewPipe's future design and development as well.

[opusforlife2]

Thanks for the interest, guys. The first step is to join the IRC from the ReadMe.

[medavox]

I work as a professional Android developer. I was the second or third ever contributor to NewPipe back in the day --
I was the first to implement the ability to play videos in the background, with your screen off (though I'm sure it's been rewritten many times since then). I've spent much longer using this incredible app.

As someone who's worked on large legacy Android codebases, I recommend against a complete rewrite.

Android Studio has great tools for converting Java code to reasonable Kotlin (which doesn't need much, if any tweaking by hand), and Kotlin's interoperability with Java is great.
You can literally convert one Java file at a time, and keep releasing in between conversions.

I migrated a large closed-source app this way myself over a period of months.

As for Compose, I've not used it much myself yet, but from what my colleagues tell me,
it's also relatively easy to have new Compose layouts live alongside existing View layouts.

A big rewrite is nearly the same amount of work as went into writing the original app.

And although you've already got the user requirements captured because you're mirroring the functionality/design of the existing app
(although you might even decide to improve things in the new app as you build it, which would make things even harder),
you haven't got the technical implementation planned out, (the existing implementations are bad, but how exactly would each one be done better?)
and you would still have re-implement all the features that were built over the last 7 years by countless contributors, not all of whom will be helping in the rewrite.

I just don't think it's a feasible amount of work.

As a compromise, you could completely rewrite the parts of the app that are the worst currently (worst performing, most buggy, worst for devs to work with, whatever you choose).

A caveat to my experience: I've worked on big Android apps,
and I've contributed work on open source Android apps,
but I've never contributed a significant amount of work on a big AND open-source Android app.
Maybe that somehow affects what is easy and what is hard to achieve. I don't know how it could, but it might.

All I know is that whether it's an open or closed source app, developer time is always a limited resource.
Choosing the path that takes less developer time will always be the more successful path.

Of course there is the fact that working on the existing codebase is becoming rather difficult --
but that's the real difficulty: finding a balance between satisfying (but ultimately unfinishable) rewrites,
and practical (but working) slow piecemeal improvements, which satisfies all partes.

Another suggestion I would have is to see if you can convince some of the former disgruntled devs (who now maintain their own forks) to come back and contribute to the main app.
If you give them more freedom to work on what they originally wanted before they left, you may also benefit from their passion and ideas, and they may fix other things along the way (I'm not hinting at anyone in particular, and I'm not one of them. I jsut read through some of the most-commented issues a while back and was sad at how many good feature PRs were ultimately left to languish. Very demoralising for a dev when that happens).

Passion to develop sounds like the real chokepoint here -- if you can make it fun to work on newpipe, you will naturally attract devs.

A shiny new codebase is definitely the right end goal, but going about it by a big rewrite may leave you stuck both
a) maintaining a legacy codebase while explaining the lack of new features to unhappy users, and
b) furiously working on an unreleased mishmash of everyone's favourite ideas

for literally YEARS.

[medavox]

Whatever you decide though, it's your choice, as you'll be the ones doing the work. And the right to complain about the final decision should be reserved for those willing to put the work in, either to help realise that choice or to build its alternative

[medavox]

And remember: shipping is a feature. A janky old version in the hands of your users, is still better (in their eyes, and therefore the eyes of potential future contributors) than some theoretical half-finished shining beacon of perfection. Perfect is the enemy of good.

[zedlabs]

some great points here also from my experience a perfectly clean codebase is never achievable. We are making an android application some amount of hackwork or workarounds are always required ( even more so in the case of new technology like compose)

[wb9688]

I think that a complete rewrite of the app is simply infeasible. Some parts could use a full rewrite (e.g. just get rid of GigaGet already), while other things can be gradually improved. I also think that the extractor is mostly fine, except for Info that should just go away, and it could be gradually converted to Kotlin.

[Fehertukor]

I think a full rewrite would be nice, but:
An other side, that few of us mention, that
You are human with a life (or without).
Have you calculated how much time it will take?
Even the basic functions to implement from the ground up?

You can't, and should not do it day and night, because burn out will get you.
(Without newpipe we are doomed/mostly on Android/).

There are solutions:
Some developers could go full time(or partially) but crowdsourcing would be not enough.

The only viable solution is Louis Rossmann and his FUTO program. I know it is not a miracle but I am sure that He would gladly help with money and what is necessary.
(He already posted on reddit ) that He would love to help the project.

If you don't want to ask him just tell me and I will send a message! (I don't know him, but I can write.).

[zestygrass]

Definitely get in touch with FUTO and Louis Rossmann. This project can use some publicity and financial support.

[teaalltr]

Sidenote, @Fehertukor @Stypox @zestygrass FUTO is going to release a new YouTube/many stuff client soon

[Fehertukor]

@jxxp9 it is already a thing.
Grayjay the app. It is quite nice. Give it a try.
For now newpipe stays but I will try this new one too.

[snaik20]

The Universal Video Feed app is listed as coming soon https://futo.org/projects

[Fehertukor]

@snaik20
https://www.youtube.com/watch?v=2jyRhiP2Oso
Check the description for more.

[trainman261]

My primary concern with an entire re-write is that the energy slowly runs out before the rewritten app is ready to be released. Users start abandoning the old app because it's not working well anymore and it feels abandoned to them (and it's obviously much easier to lose users than to get them back). Which on the other hand disincentivizes developers.
An approach that I've watched work much better (albeit in a non-FOSS-setting) is to look at what modules a new app would need, and then work on one module at a time, starting at the backend. Get the module up and running, polish it, then replace the old code with that module. In the beginning, there won't be all that much visible progress for users (and the amount of bugs or especially performance issues may go up in that time), but visible progress will keep increasing, and eventually the stability as well. Some already really disgruntled users may leave, but carefully managed, most will stay. It's no silver bullet, but maybe a good middle ground where there are no perfect solutions.
And honestly, that sounds fairly similar to what you've tried with the player, and my experience with that approach seems to mirror that. So, you're on the right path IMO. Yes, there are weird bugs because of the friction between new and old components - but that's to be expected, and will eventually improve. And as a NewPipe user, I have to say, the player works better now for me than it did previously. Kudos!
[Unfortunately, my experience in coding for Android is pre-Kotlin and pre-Material design, so I'd have to get into that first, and personal resources are lacking at the moment. I'd love to support this project though, maybe in a year or so.]

[dvdkon]

The problem is, is the current code modularised well enough for that to work? I don't have any experience with it, so I can't say, but theoretically ideal approaches can run into a lack of clear interfaces where modules could be replaced. If each new module rewrite needs to rework 50% of the remaining code, is it worth it? This approach also doesn't allow replacing the app's fundamental structure, which might be needed.

[trainman261]

In the case I had in mind, the code was pretty much not modularized at all - written in smalltalk in the 90's, and a massive codebase.

If each new module rewrite needs to rework 50% of the remaining code, is it worth it?

I don't have any experience with the NewPipe code base either, but I wouldn't be surprised at all if rewriting the entire code base took less time (possibly significantly less time) than writing new modules and integrating them. My concern is: how do you keep the users on board while you're rewriting everything? It will take a long time. As others here have mentioned: there are competitors. (One could of course ask the question if it would be better to move development efforts to another similar open source project and join forces to create a better app, but that's outside of the scope here and I personally use NewPipe because it in my experience works better than other open source alternatives.) If the existing app doesn't see any or enough proper maintenance, how many users will we retain? I'm afraid many will leave. And if the project dies because of that, it doesn't matter anymore that in theory the "rewrite the app entirely" approach is more efficient - it's dead.

EDIT: an additional note: unless I'm understanding something wrong here, NewPipe development requires a fair amount of firefighting, because Google makes changes every here and there that break NewPipe, obviously without any advance notice. If this weren't the case, that would make a much stronger case for an entire rewrite.

[zabihjan]

The app has gone through many hurdles and it has become a solid app as a user's point of view. I have seen many apps rewritten from scratch such as sky and libre, they are trying their hardest but it is super slow progress.

[dyc3]

As the maintainer of Opentogethertube, and a fellow student with a life: Do not rewrite it from scratch. Upgrade it piecemeal. Don't just throw out battle tested code outright.

OTT has seen it's fair share of module rewrites, and there are always bugs that come up as a result. But, it's better to find and fix those bugs early on. With a total rewrite, your users are just going to find all those bugs all at once, and that's only if you manage to complete it. (Of course it would be better to not have bugs at all, but that's unrealistic.)

For example, OTT recently upgraded from Vue 2 to Vue 3. The details aren't important. I slowly tweaked and shifted things around to prep the codebase for the switch, and eventually I was able to do it in an admittedly fairly large, but much more manageably sized PR.

Large rewrites are exhausting, and it's so much easier to miss stuff.

[rahul-gill]

• new rewrite may take long time. also "modern" doesn't stay "modern" for long. modules is a good idea

  • Contributing to smaller, newer codebase is easier for new contributers but we can do somethign similar with modules. Some people may have experience with ui development but not with player, so if they want to contribute they can focus their effors on the ui module only.
  • Also modules will allow to improve a specific part in isolation if needed in future. It would be easier to work on different modules on different branches without causing big merge issues. We can refactor some of them and rewrite a few that require it.
  • It would be good to use the idea of state and logic modules(logic module are only like input-output, with consistent interface and can't consume state modules and state-modules contain state. data models can be put in any of these as we see fit to what the data-models are more related)

• material3, jetpack compose is ui only thing so they can be handled independently of other things, but kotlin conversion is required in other modules also because otherwise in the kotlin ui, we'll have to deal with lot of nullables because of java

• I think kmm would not be good choice because that would make things hard for people without macs(like me).

• as for compose, the performance difference is visible in cheap phones and the workarounds required for parity in the performance will hurt the easyness part of compose. Some apis are new and experimental in compose. But I do belive the code overall looks better with compose and it'll invite more contributions. Since there's too much confusion on this side, I'll suggest to focus on the non-ui parts of the app first.

I'm in for contribution.

[opusforlife2]

the performance difference is visible in cheap phones and the workarounds required for parity in the performance will hurt the easyness part of compose

Is this measured somewhere on a blog or website? Or is this anecdotal evidence? If it's anecdotal, how recent is this?

Our minAPI is Lollipop and will probably stay that way for a long time, so if this hurts performance badly we might have to think on this.

[opusforlife2]

Oh, and thanks for offering to contribute!

[AudricV]

Regarding Jetpack Compose's performance, Android teams want to keep improving the UI's toolkit performance in the future.

I strongly recommend you, if you haven't already, to watch and/or listen to the episode 199 of Android Developers Backstage's podcast (video with audio link, audio only link), where the Android teams working on Compose explain this topic in a very detailed way.

Its TLDR from my point of view: if we use the good practices to build the app's UI with Compose, we will probably just have to update the Compose version we use to benefit of most future performance improvements.

[ticmaidev]

Most compose testers back out only for the reason of dev builds giving poor performance

this is the dumbest thing to say and the fact that people keep repeating this over and over bothers me a lot.

do you really think any software developer (testers lol) is not aware that debug builds are slower? or is it something only you ever realized?

jetpack compose might not seem slow if your end users are people from first world country with decent phones. performance becomes an issue when devices are cheap and this is a fact.

[AudricV]

As noted in the blog post of 0.26.0 and 0.26.1 releases, here is our final decision:

It’s been made plenty clear that replacing parts of the code piece by piece is a much better way to proceed, as it will let us keep releasing new versions, as well as keep the community in the loop.

We will so proceed with the incremental option, and rewrite progressively the app.

The complete rewrite option would have been really hard to do, as we currently don't have the resources to do so.

Like I wrote above, some parts needs to be replaced and thought again entirely, like the player.

Unfortunately, the team isn't currently able to handle the rewrite now, if you read the State of the Pipe 2023 blog post, you may have understood that the team is in a lack of maintainers.

Two developers who are currently pretty busy won't be enough to make the rewrite progress in a reasonable way, we need more active developers (it would be awesome if we can get the double or the triple of the current amount). So like written in the post, "[i]f you want to step in and help [us] with maintaining the app, e.g. reviewing PRs, developing code or doing community management, please get in touch!".

We will let you know when we start the rewrite of NewPipe, but you can already help us by trying to provide fixes for bugs that are in the current app. Remember that we stated in the post of this discussion and in the blog post I linked above that we still have a few major releases to complete before starting the rewrite.

Thank you for everyone who made a constructive feedback, it is invaluable!

[Chinaqth]

Count me in, but I am currently working on HarmonyOS Develop, May not have much time on this

[acrodemocide]

Most of my background is in full-stack web development, but I've done a little with Android (looking to learn more and expand my skills). I love this app and it is my sole app for viewing YouTube videos, so I'll be coming on and helping review pull requests and potentially help with bug fixes and new features.

[fuzzblob]

I've been meaning to reply as well but was nervous as I have virtually no android Dev experience. But after having implemented a basic app in Jetpack Compose the other day my confidence that I can make contributions has gone up considerably.

My background lies with video game development and I want to see NewPipe flourish 😊

[opusforlife2]

Status update: we currently have a release candidate out for 0.27.0. If you wish to contribute small, easy-to-review bug fixes, that would be great. If you wish to wait for the rewrite efforts to start contributing, that's fine too.

[GGindin]

I'd love to give this a try! Going to look at some issues for now.

[XilinJia]

I'd love to do some work.
How about we first migrate the code base to Kotlin? I would like to put some effort do that.

[XilinJia]

Sure, will check on some PR's. I look forward to migrating onto Kotlin sonn, if that's agreed to everyone.

…

On Mon, Mar 4, 2024 at 10:35 AM opusforlife2 ***@***.***> wrote: @snaik20 <https://github.com/snaik20> @uragiristereo <https://github.com/uragiristereo> @imashnake0 <https://github.com/imashnake0> @lm41 <https://github.com/lm41> @YongJunLim <https://github.com/YongJunLim> @AaronRietschlin <https://github.com/AaronRietschlin> @sandy-8925 <https://github.com/sandy-8925> @PrimoDev23 <https://github.com/PrimoDev23> @EricDriussi <https://github.com/EricDriussi> @rahul-gill <https://github.com/rahul-gill> @Profpatsch <https://github.com/Profpatsch> @Chinaqth <https://github.com/Chinaqth> @acrodemocide <https://github.com/acrodemocide> @fuzzblob <https://github.com/fuzzblob> @XilinJia <https://github.com/XilinJia> Hey guys. You've been pinged because you've all volunteered to help out with the rewrite at some point or the other. If you guys have lost interest or are otherwise busy with other things, that's absolutely fine. But if you have the time and the inclination, we invite you to help out from now on. As Audric mentioned, we currently have just 2 core developers, both of whom happen to be pretty busy right now, so please don't open any giant or complicated PRs. Instead, it would be super helpful if you helped review the current pending PRs, in order to get them merged faster. Small bug fix PRs are also fine, in order to familiarise yourself with the codebase. Even if you're not currently interested or able to participate, please leave a reply so that we can get an idea of how many people we're working with. Thank you for volunteering and helping keep Newpipe alive! ❤️ — Reply to this email directly, view it on GitHub <#10118 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPXXQIORRALHUSA3AYZGCDYWQ56FAVCNFSM6AAAAAAYP4FGV6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DENJSG4YDI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[chriss2401]

Hi! I have no experience with app development, but I could help with the extractor if there is re-writing/porting needed there. Let me know if you need an extra pair of hands. I used to write java, but have written a lot of C# recently, so I think the transition wouldn't be that hard. Would just need a kick start with the code base.

[GGindin]

Hey everyone! I'm a relatively inexperienced dev overall, but I have some experience writing Kotlin. I use NewPipe nearly every day, and I'd love to be able to find a way to contribute to the migration. Is there anyone willing to help point me in the right direction so I can get started? I'm pretty new to open-source contribution.

[opusforlife2]

You can join the Matrix room from the Readme. Right now, the focus is on eliminating all the backlog so that we can proceed with the rewrite without any prior baggage. Small bug fix PRs are welcome, but feature PRs, or big hard-to-review PRs will likely be put off for now.

If you're interested in working on something, come join the chat and let us know!

Also, are you interested in doing drive-by contributions or do you want to help with the rewrite effort in general? I could add your name to the list here #10118 (reply in thread) if so.

[andreluizreis]

Hi, I am a experience Android developer (more than 10 years) and I discovered this thread now. I started using NewPipe in 2023 until now.

I work with huge apps and I am working on a similar problem like here. Huge app done using Presenters binded to their respective Fragments.

I would say that is impossible to rewrite the entire code, or start from zero. The app should keep both architectures and you can migrate by pieces.

First, you decide how you are going to split into modules and you rewrite the code of each modules over time.

I am working with MVVM, Kotlin, Compose, ViewModels, Coroutines, Workers, UseCases, Repositories, etc.

It is not easy and it takes a lot of time.

Pay attention that now Apple could be forced to open the iOS ecosystem. So it is interesting to take a look at the Kotlin multiplatform project. I think it will be a powerful tool and it will grow more overtime. You can structure your app to work well with Android and have a good portion done for iOS as well. Don't waste your time thinking about Flutter.

I would like to give more details about the structure but I don't have too much time.

Try to keep one small responsibility for each UseCase and keep in mind that the UseCase needs to be modular and isolated. It will be easier to maintain and test.

Use Mockk for testing. Really good.