Iceland is a great country to visit with kids because its relatively safe. There are few of those kinds of humans I call "wolves" and not a single incident of missing children that I know of. I have a six year old daughter who is never out of my sight when we visit California, but who is free to roam around without me here in Reykjavik, which we both consider a great privelage. There may not be a ton of attractions and flashing lights, but you don't have to have to panic if you get separated from your little ones. Tell them that everyone understands English if they get lost and need help.
For older kids you might try giving them a couple of thousand kronur (about the same as a twenty) and setting them loose on Laugarvegur, the main shopping street in Reykjavik. There are cool music, clothing and book stores, plus hot dog, ice cream and candy shops. Set up a rendezvous location like Austurvöll, a cute city park downtown that's very popular in summer (turn left off Austurstræti at Pósthússtræti) and be free! Warning: Stores in the dowtown area are only open from 10 am til 2pm on Saturdays and are Not Open on Sundays !! Takes me by surprise every single weekend!
Of course there are the swimming pools, super popular with the young set, and also the Zoo and Family park (Fjölskylda-og Húsdýragarðurinn) in Laugardalur. The zoo is quaint, with a bunch or farm animals and some seal, reindeer, mink, a turkey or two and a bunch of hands-on stuff and things to climb to run off extra energy. It's inexpensive and very countryside in the middle of the metropolis. Check it out.