"The traffic and parking impacts existing at both the Scots Bellevue Hill[senior and junior] campuses are well beyond unacceptable,"their submission said.
The group feared the project would exacerbate congestion and parking shortages due to rising enrolments at the college,which has previously fought toincrease its student numbers.
An objection was also received on behalf of Mr Joye,who owns the suburb's sprawling Barford estate,over concerns the project would impinge on the property's sweeping harbour views.
But,although the Victoria Road trophy home was deemed the most affected by any view impacts,it would suffer only"minor loss of distant water views"towards Rose Bay,with the biggest impact to views from the driveway and garden terrace,the department said.
"However,the vast majority of harbour,distant and district views from internal habitable areas would be unaffected,"the department said in its referral.
Woollahra Council was"generally supportive"of the proposal. But it said fixing the traffic problems was of"primary importance"and echoed concerns the school was breaching its cap of 1120 students.
"This has direct impacts on traffic and parking problems in the vicinity of Scots College,including arrangements for the drop off and pick up of students during peak times,"the council said.
In its response to the submissions,Scots College said the makeover would have"no impact on access,traffic generation or car parking".
"No increase in student numbers is sought by the subject application and the college is in course of preparation of a concept development application which will address this issue and seek approval for a new masterplan for the Victoria Road campus of the college."
The college said it had lodged separate plans with the council for an 80-space car park beneath the school's tennis courts and an additional student drop-off area on the western side of Victoria Road.
If approved by the commission,the revamp is due to be completed in 2021. The college said most of the $28.86 million project would be paid for through donations and fundraising.
The NSW Heritage Council submitted the"complicated and more massive roof form,with its high turret and dormer protrusions,and prominent balustrading to the rooftop terrace"would"overshadow and diminish"the school's neighbouring 1883 boarding residence Aspinall House.
But Scottish architectural expert Alastair Disley,who was a consultant on the proposal,said the complex roof form was"essential".
"The integrity of Scots'Baronial design depends on its faithful replication of historic roofline
elements - typically from 15th to 17th century Scottish castles - to balance the more modern
wall elements below - such as large 19th century plate glass windows,"Dr Disley said.
The school's website says the design concept reflected"a fortified building,a defensible structure,a safe stronghold;in short,a sanctuary. In the Scots context,it embodies a safe place for boys."
The Department of Planning recommended the building's approval on the basis it was in the public interest to deliver improved educational facilities. It said any traffic and parking problems could be managed and the project would create more than 400 construction jobs.