Usually they are well-meaning when they give you annoying advice like, "Maybe you should pursue something more sensible." Or "I don't think there's really a market for what you're doing." Or even "I hate to see you wasting your time on this."
Usually it's a waste of breath to argue with them. The best answer may be, "You may be right, but this is my passion. I need to do it, if only for myself." They still won't get it, but they may stop imposing their viewpoint on you. Maybe.
What they say to you isn't as important as what you say to yourself. Unfortunately, sometimes their doubts are infectious. If you feel that starting to happen to you, this could be a good quote to remember:
“Only those who risk going too far can possibly find out how far one can go.” - T. S. Eliot
You may be risking going too far.
You may be exceeding what you can do.
You may be coming up with something that is ahead of its time.
You may be creating something nobody wants.
On the other hand, you may be creating something people have been waiting for without knowing it.
The only way to find out is to risk it.
What if the outcome isn't what you hope for?
Will you enjoy the process, or at least a significant part of it?
Will you learn something valuable along the way?
Will you be stronger for having--and surviving--the experience?
If so, will it have been wasted?
Now...go for it! T. S. Eliot and I are on your side.
(For friendly guidance on taking your project from idea through to publication get my book, Your Writing Coach, published by Ncholas Brealey and available from Amazon or your other favorite bookseller.)