Yes, for depleted Uranium the metal itself is toxic already. The radioactivity is less of an issue with depleted uranium, hence the depleted. This is similar to other heavy metals not being healthy. You don’t want that in your body.

The question is how much depleted uranium ended up where? The bunker buster bombs penetrate fairly deep underground and most of the Uranium should end up there as well. The site of Nasrallah’s bunker was dug up and thus the material spread among the excavated debris.

Now that this danger is known, decontamination and encapsulation can be used to mitigate the consequences.