He certainly wasn't destroyed. Why not? Because Joss Whedon, one of the most iconic writers/directors ever, would have given him a more memorable death.
The camera just sort of pans away and a slight yellow flash is seen. Notice what I said there "slight"? It wasn't a harsh light usually meaning the destruction or smiting of something/someone.
Here are some examples of what I'm talking about.
No, the light was small, slight, and only faintly seen. Not to mention the fact it was also yellow, the color of the Mind Gem, so it's understandable for people to assume he was destroyed.
However, Vision himself said earlier in the film that he had no intention of destroying Ultron as he saw him as "unique".
This leads me into my conclusion: Vision simply reabsorbed Ultron back into the mind gem.
The light, as I have mentioned earlier, was small. It was more like someone was being transported, ascended or... absorbed. Well, Ultron's brain waves were designed off of the interior of the Infinity Stone. He also showed the ability to have his consciousness leave his body, so in this case, maybe he did the same thing, he just went into the Mind Gem.
Further evidence, Joss Whedon would have written something more dramatic or something with a zingy one liner or dramatic line like Wanda ripping out Ultron's "heart".
It should have been more dignified or understandable. It should have been a moment where his, Ultron's, death should be understood, resolute and lasting, so he would never be able to come back.
They didn't give us that. Instead, they panned the camera away, leaving Ultron's exact fate unknown.
Oh how I giggle at such a thought.