Told I could not get to scratch

Kinda want some opinions on this. Have only played golf for 6 months however have gone from high 120s to low 90s high 80s in 3 months as a member of my local club. Playing with a guy and mentioned I would one day like to get to scratch. He instantly shut me down saying it wouldn’t happen(bear in mind he plays off 28). My arguement for this was that I was highly sporty and played at academy football/soccer level. And was a national champion boxer 3 years in a row and that my dedication to sports alone would get me there. I have a very free schedule the next few years so my question is. With 0 time limits could I get down to scratch in 3/4 years.kinda knocked my confidence so wanted a Reddit opinion. Thanks guys :)🏌️‍♂️