There have continued to be connections between the show and the movies the whole time.
The Dark World aftermath cleanup was as boring at that movie itself, sure. But Lady Sif from the Thor movies showed up in a few other episodes too.
There was of course the huge and awesome tie-in with The Winter Soldier and the end of season 1.
The alien whose blood resurrected Coulson before the start of the show was a Kree, and the Inhumans were also created by the Kree, tying in to Guardians of the Galaxy which introduced the Kree into the MCU.
There was some big secret project that was causing drama between the different surviving factions of SHIELD in the lead-up to Age of Ultron, which turned out to be the helicarrier that saved the day at the end of that movie. SHIELD's hunt for the remnants of Hydra also tied in to the discovery of Loki's scepter that kicked that movie off.
I don't think there was any connection to Ant-Man anywhere.
The Sokovia Accords introduced in Civil War impacted the superpowered Inhumans etc on SHIELD's team.
It was a less direct, more thematic connection, but at the same time Doctor Strange was introducing magic into the big-screen MCU, Agents of SHIELD also had a magic-related plot (lead into by the Ghost RIder subplot), and the visual effects used for the casting of magic in the show are obviously the same effects used for the casting of magic in Doctor Strange.
I don't think there was any connection to Spider-Man or Black Panther either.
The Kree feature prominently in season 5 of SHIELD too, tying in further with the cosmic part of the MCU, Guardians and now Captain Marvel with their Kree-heavy settings.
And of course, in the big finale of that season explicitly refers to the attack by the Children of Thanos on New York that was at the start of Infinity War.
So they've been making it pretty clear all along that this is meant to be the same universe, and the show's creators claim
that it is meant to still be in the same universe, just "pre-snap", and that they have some way of reconciling how that's possible that they're not spelling out in the show just yet.
I'm wondering if it will somehow be connected to how the current events of the show seem to involve more than one faction of people that seem to be hunting down people who are somehow or another not supposed to exist in this timeline. They haven't really clarified exactly what's going on there yet, but it makes me wonder.
ETA: Meanwhile, nothing happening on SHIELD has been big enough that it should have warranted the attention of the Avengers. For seasons 2 and 3 they were mostly dealing with Inhumans, the remnants of Hydra, and and the legal status of SHIELD in the post-Winter Soldier and then post-Civil War world, nothing really world-ending. Season 4 was a bunch of smaller plots mostly only affecting the SHIELD crew themselves: Ghost Rider, LMDs, and the Framework. Season 5 was a literally world-shattering plot, but most of it took place in the future, and by the time it was back in the present and things were getting big enough that it might warrant calling in the Avengers... they were simultaneously busy fighting Thanos, a much bigger threat.