Migration to a strong OOPs based structure (like Nest.js)

[Manik2708]

Currently the structure of our code is not very rigid and testable. We are not facing too many problems because the number of external services are less. Like we are using only Mongoose which don't require Dependency Injection to test, but at the same time we have Redis which is not being tested because we don't have DI in our codebase. We expect the app will expand in future and more services will add to the codebase. Addition of more services and features can become messy if not tested. Unfortunately Nodejs don't provide a strong structure and hence DI is very tough in it. Packages like tsyringe are buggy. So I suggest to move to a move to a stronger framework used by many closed-sourced organizations like Nestjs. This framework is very similar to Node, therefore migration will not be that tough. It will provide OOPs and class based design pattern used by many big scaled organizations using java as backend. Maintenance, re-usability and testing would become easy by this.

[hg6658]

i think giving DI over mongoose sounds great then we can use "mockingoose" library to mock a resource/model without ever touching production database

[Manik2708]

1. There is no need to change client UI. But if Modular Structure is required in client, then Angular is the best framework.
2. The core of Nest is well documented, and talking about packages, all the node dependencies can be installed on it. The GitHub repository of Nest is active, so issue can be reported easily. It's just a wrapper of Nodejs with decorators.

[Anubhav-2003]

Having modularity in our codebase is a must. I have had extensive discussions on this before, and I think this is really important. Abstraction and encapsulation makes contributing much easier for new contributors, and at the same time makes the system design much more production worthy.

[hg6658]

so what should be our next steps.

[Anubhav-2003]

I say one of the first things that we should do is getting the user authentication, its own micro service in the API. GraphQL is definitely not meant for user authentication. in fact, we had a few issues for unauthorised access. Now we can decide upon how we should implement the authentication. It should be a priority in my opinion.

[pranshugupta54]

Plus one for authentication, I've mentioned this multiple times on Slack that it should be the priority. We're having too many security issues, allowing easy access to restricted screens. We can always change the code structure later but should focus more on important things like security.

[pranshugupta54]

We had some previous discussion on Object Oriented too - @xoldd
#2011 (reply in thread)

[xoldd]

I still think this will be a worthless endeavour.
Why nest.js is bad
Why OOP is bad

In graphQL servers dependency injection is accomplished using the graphql context that is available to every graphQL resolver and has the global lifetime of an http request. Although it is not an advanced all encompassing form of dependency injection, it gets the job done for most use cases.

The core of Nest is well documented, and talking about packages, all the node dependencies can be installed on it. The GitHub repository of Nest is active, so issue can be reported easily. It's just a wrapper of Nodejs with decorators.

This is certainly not true. Nest.js is only good as long as the packages and integrations it officially supports are being used. As soon as you introduce a non-supported dependency, you'll will have to do weird hacks and workarounds to make it work in the way nest.js wants things to work.

Anyway, for a graphql server graphql-modules would be a better choice than nest.js.

[Anubhav-2003]

@xoldd I don’t really think Object oriented programming is bad. It is your POV, anybody can cite internet articles and videos, backing them up. Even I can put link of a few articles and videos which praise OOP, doesn’t mean everyone has to think it is useful. Nothing is absolute in terms of opinion. One of the main reasons you think it is not worth It, as per our last conversation, was because you thought it is more about aesthetics, than about getting actual work done. But actually, the converse is true when you have a highly modular and well structured codebase, it becomes exponentially quicker to not only debug issues, but also have separation of concerns for individual issues. Most big firms have their codebases modulared, Irrespective of whether they are working on springboot with java or not, Currently, the structure of our codebase seems more like a personal project than an actual industry grade repository. One of the first things that we should start with is getting the authentication and static file management out of the graphQL paradigm.

Then creating micro services for each individual process and then interlacing them. In fact we are using graphQL in a faulty fashion, even when graphQL is used in an organisation, it is abstracted behind a restful API. The client never directly sends request to GQL. The request object is parsed before being sent to the Apollo middleware. But in our case, the graphQL handles everything. Which is good for a hobby project. But our vision is to create a professional organisation spanning over multiple nations, therefore it is high time we start adhering to standard system designs by getting out of our comfort zone. And please don’t say objected oriented programming is bad, I respect your opinion but helping on a repository on a multinational trading organisation like groww which handles millions of transactions per year being completely run on the principles of abstraction and encapsulation, is a far greater proof in my opinion, than just a YouTube video. Thank you.

[xoldd]

One of the first things that we should start with is getting the authentication and static file management out of the graphQL paradigm.

GraphQL can handle file uploads using multipart form data if they are to be stored on the same filesystem as the server. The current implementation of base64 encode/decode file uploads and storing them in database is bad. If something like S3 is to be used(which it probably isn't), then file upload doesn't belong in the server and it should be handled on the client(client -> s3) directly to not have to incur extra ingress/egress(client -> server -> s3).

In fact we are using graphQL in a faulty fashion, even when graphQL is used in an organisation, it is abstracted behind a restful API. The client never directly sends request to GQL. The request object is parsed before being sent to the Apollo middleware.

The client does directly send the request to GQL. Unless you're confusing it with persisted queries?

But in our case, the graphQL handles everything. Which is good for a hobby project.

Why should it not handle most of the things? That is the point of using graphQL. Resolver functions are simplest form of abstraction for a graphQL server and we are talking about a monolithic application here. Having 10 nested layers of services won't make it better.

but helping on a repository on a multinational trading organisation like groww which handles millions of transactions per year being completely run on the principles of abstraction and encapsulation, is a far greater proof in my opinion, than just a YouTube video

Your argument sounds like this:- "Banking software written in COBOL scales to billions of users therefore COBOL is good and everyone is wrong about COBOL being bad." I'd rather not discuss about this anymore because from your argument I feel like even if I cite well respected academic findings about it your opinion won't budge. One thing I'd like to say is that OOP has both good and bad aspects, and when I say OOP is bad, I'm criticizing those bad aspects(easy vs hard vs simple vs complex). I wasn't meaning to argue about this with you again, was just sharing my opinion on this discussion. Remember nest.js is angular.js for the server side, and angular lost to react.js in the frontend world 😉.

One more thing to keep in mind is that Peter has reiterated multiple times that the costs of external services should be kept minimal, everything should be done in house if possible. So, far the only exception is the database. So, the talk about microservices, scaling to billions, modular system similar to big firms etc., doesn't make sense for the current requirements. At that point there'd be full time professional engineers working on this project. Requirements change over time, software changes over time, you don't try to build the next google on the first try, it doesn't work that way.

[Anubhav-2003]

“ I'd rather not discuss about this anymore because from your argument I feel like even if I cite well respected academic findings about it your opinion won't budge”…… well just like that, I can also cite a lot of academic finding that discuss the benefits of OOP, in-fact there are far more of them in the industry and internet, better enough I can get actual testimonials from software engineers as well. But what would that proof? Nothing. Citing articles from the internet means nothing. Every argument has two sides of the coin, and the internet will be filled with both.

“ GraphQL can handle file uploads using multipart form data if they are to be stored on the same filesystem as the server. The current implementation of base64 encode/decode file uploads and storing them in database is bad. If something like S3 is to be used(which it probably isn't), then file upload doesn't belong in the server and it should be handled on the client(client -> s3) directly to not have to incur extra ingress/egress(client -> server -> s3).”…… I have been reiterating over it for a long time that if we want to become a social media platform for community organisations, then we must completely refactor the present static file handling architecture…. This is part of GSOC so that should be sorted.

“ Why should it not handle most of the things? That is the point of using graphQL. Resolver functions are simplest form of abstraction for a graphQL server and we are talking about a monolithic application here. Having 10 nested layers of services won't make it better.”……… Our application is far from being monolithic, further why should not it handle everything? Because it is inherently not optimal for everything. If that was really the case then all organisations using graphQL would simply create one instance of apollo server and run everything on it. The biggest of them being security. Different API paradigms have different purposes. The main purpose of GQL is to give more granular control to the client for fetching data in a dynamic fashion. It does not mean one will run an entire api based on it just Because you can. This is not a project to demonstrate graphQL proficiency but rather have professional code. Then what is the purpose of having other approaches like REST.

“ One thing I'd like to say is that OOP has both good and bad aspects, and when I say OOP is bad, I'm criticizing those bad aspects(easy vs hard vs simple vs complex).”…… And completely ignoring the good ones? So then I should say when I am advocating OOP I feel that the pros outweigh the cons.

“Your argument sounds like this:- "Banking software written in COBOL scales to billions of users therefore COBOL is good and everyone is wrong about COBOL being bad.”.……………. Contrary to this statement about COBOL, majority in the industry actually advocate OOP. Your argument is the unpopular opinion in general. Also by this logic should all good practices in coding be given up. OOP only brings complexity for people who have skipped CS fundamentals. Otherwise most big organisations wont prefer it, if it introduced complexity, they appreciate the abstraction and separation of concerns. Reduces debug times significantly.

“ One more thing to keep in mind is that Peter has reiterated multiple times that the costs of external services should be kept minimal, everything should be done in house if possible. So, far the only exception is the database. So, the talk about microservices, scaling to billions, modular system similar to big firms etc., doesn't make sense for the current requirements.”…………. Most of the micro services discussed here are going to be in house implementations. So having multiple of them has no correlation with the cost nor resources as all of them would still run on the same instance consuming almost similar resource . It’s more about having modularity in the sourcecode. Costs would not increase unless usage increases, in which case it would increase irrespective of architecture.

“ So, far the only exception is the database. So, the talk about microservices, scaling to billions, modular system similar to big firms etc., doesn't make sense for the current requirements. At that point there'd be full time professional engineers working on this project. Requirements change over time, software changes over time, you don't try to build the next google on the first try, it doesn't work that way.”…………. Does that mean no company smaller than google practices good system designs and practices?? Forget google, you mean to say there are no other OS organisations that have OOP. That is just an excuse, traffic has nothing to do with the architecture, you don’t wait for a million people to visit your website before you say “let’s have good code now”.

Well I cant change your opinion, you can’t mine. There is no point arguing. We should respect each other’s opinion, and let the community decide what is best. After all this is opensource. We head the way what the majority feels. And I feel it needs new architecture.

Thank you.

[palisadoes]

I appreciate the vigorous, respectful discussion. It’s important to have debate, especially as we’re going to need to chart the medium term goals of the various apps in the near future.

Remember the code of conduct. We are close to the line.

[palisadoes]

I plan to listen to the YouTube video to understand the context of OOP dissent.

[Manik2708]

I was going through this conversation and I think that both sides are right in their own way. Real world also have these kind of tough decisions. I think we should consult some more experienced developers and ask about their views on migrating to OOP. In the end, even if we migrate to Nestjs we will find some things which are better in Nodejs. We should look to our priorities and needs of the project. I started this conversation because I recently migrated one of my personal projects from Node to Nest and things were seeming fine to me but a different view of @xoldd made me to rethink about my decision. I hope this conversation will end with a decision.

[palisadoes]

The tone of the video gets progressively less extreme. He eventually has some common sense suggestions.

In the beginning:

1. He feels a that the exclusive use of classes not a good idea. He recommends using combination of functional and object-oriented for high level code when you are not worrying about efficiency:
   1. https://youtu.be/QM1iUe6IofM?t=370
2. He’s against object inheritance and possibly its tight coupling. I decided to lookup the current view against inheritance, and discovered the current thinking using object composition which looks a reasonable approach.
3. He feels that poor OOP implementation of abstraction needs to be policed, I’m assuming he’s concerned about nesting of objects which makes troubleshooting and maintainability difficult.
4. The biggest criticism of OOP is encapsulation
5. He criticizes the concept of abstraction being interpreted as simplifying complexity
   1. https://youtu.be/QM1iUe6IofM?t=1790
6. He is against creating tiny objects that create difficulty in interpreting the code base
   1. https://youtu.be/QM1iUe6IofM?t=1903

Then he makes the recommendation that if OOP is to be used, then simplify the code base by:

1. having a mix of objects and functions

   1. https://youtu.be/QM1iUe6IofM?t=2003

2. minimizing the number of objects by having more complicated methods

   1. eliminating the tight coupling of object inheritance and use composition instead

3. maximizing the number of easily readable functions

   1. using clear comments

      

   2. using nested functions

      

   3. doing this stuff

      

4. reducing the code file count but not making the files too big

He then lists his recommendations, and these points were not mentioned earlier in the video

1. pass global variables / state as parameters (obvious)

2. consider packaging global variables into classes

   1. https://youtu.be/QM1iUe6IofM?t=2245
   2. Here is the summary image

   

I'm not versed in the various Javascript flavors, and don't code in it. This is why I try to challenge you to justify your assertions to evaluate the merits.

Based on the video, I don't see a compelling reason to migrate to a superset of Typescript. We already suffer from package deprecation fatigue. Exacerbating this by hiding complexity under abstraction doesn't seem to be wise.

He admits that:

1. OOP isn't ideal, but often necessary.
2. When OOP is used, be judicious and avoid inheritance

It's not a controversial approach.