As @Do_Little said, Frigates have advantages - they have bonuses for a range of capabilities that make them more able in certain roles: The Inquisitors are better at supporting a small gang than a destroyer would be, the Executioner is better at tackle and controlling an opponent than a Coercer would be and so forth.
Destroyers are good at one thing: anti-frigate work where they can engage at range (if they can’t engage at range then things get a bit messy). So, in a fleet they are going to have the role of killing the tackle frigates that are trying to pin down the larger more valuable ships. T2 Destroyers have a different role, but that’s the specialised nature of T2 ships. T3Ds are capable generalists.
The problem a destroyer has against “big ships like cruisers” is that a Destroyer is a relatively big ship, slow ship as well . “Size” is represented by the Signature Radius, typically a Destroyer has twice the radius of a Frigate, and a Cruiser is twice the radius of a Destroyer. Additionally the speeds of the Cruiser and Destroyer are broadly similar making range control a lot harder.
Against a Battleship you’ve got to be very wary of the light drones they will almost inevitably throw against you. A Destroyer is more vulnerable to light drones because it’s relatively slow and large compared to a tackle frigate.
You are a capsuleer: the feeling of immortality is understandable. I like destroyers, but they are not invincible. For many things a frigate is the best tool.