The Government is being urged to devise an action plan on the issue of older driver safety, with over four million drivers now aged over 70.
"With ever increasing numbers of elderly drivers, this is a growing mobility and road safety issue that won't go away. The government needs to act now," said IAM chief executive Simon Best.
The IAM and Vision Critical released results from a survey of 1,297 people showing that 42% of respondents are worried about an elderly relative driving but are unlikely to do anything about it.
Almost half (47%) of those who had spoken to an older relative about their driving were faced with a negative response.
With driving so closely linked to independence for the older driver, Mr Best acknowledged that "giving up the car keys can be a very stressful process".
He said voluntary online driving assessments can be a useful way for drivers to get unbiased feedback and help them to stop driving at the right time for them.
"We are finding while there are some elderly drivers who should not be on the road, most get it right and as many as 15% give up too early," he added.
RAC technical director David Bizley said: "Research carried out for the RAC Report on Motoring found that only 15% of UK motorists rate ageing drivers as a major concern. This is backed by Department for Transport statistics which demonstrate that while the number of drivers aged 70 and over is on the increase, older motorists cause fewer accidents than their younger counterparts.
"While the majority of motorists would like to see more rigorous medical checks on drivers aged 70 and over, such as regular eyesight tests, there is no strong evidence that more draconian measures such as compulsory re-sitting of the driving test at regular intervals is required.
"Drivers themselves tend to self-regulate as they get older - for example, stopping driving at night or on motorways or in adverse weather conditions - and tend to recognise their own limitations. Family members too, have a role to play in helping older drivers make the right decision regarding when they should hand over their car keys."
As well as a Government action plan and making such assessments widely available, the IAM is also calling for more car designers to consider older people's needs; better information for older drivers and their families; and improved road design to help older drivers stay behind the wheel for longer.
Copyright Press Association 2014