A shared, intuitive user interface between the debugger and profiler with a single, shallow learning curve ensures scientific developers and HPC experts alike get the maximum value from your tools investment.”
insideHPC: Please tell me about this new Allinea Forge thing. I’m really curious. What does it do?
Mark O’Conner: What does it do? So we’ve had DDT that’s integrated profiling, debugging and kind of combined interface to make DDT use for quite a while now. Forge is the new name for this combined entity – of these two pieces put together – because it’s the one place that developers can really come to make that code better. What I wanted to show you today is the new workflows that we enabled this now that we’ve added things like editing and committing source control directly from the product.
insideHPC: So kind of a unified map of the world here.
Mark O’Conner: Yeah. We resisted doing this for quite a long time because people ask and say, “Oh, I want to edit some DDT, I want to edit some MAP.” And we thought, “Well, we don’t want to make another IDE because the world does not need one of those.” But people kept on asking and we’ve been around for 10 years now – this our 10-year anniversary. What we’ve realized is that the HPC workflow is unique. In a typical enterprise environment, management can mandate that everybody uses the same IDE and the same project structure – they do all the same stuff. That fundamentally doesn’t work in HPC. You’ve got scientists from a range of different fields all collaborating together. You’ve got support staff, you have to work with them. Then you have system admins who looking at the cluster. So you can’t say everyone has to work in the same way.
A typical example of how Forge would work in this situation is that you have a user who has written his wave equation code we’re going to look at, and he’s not getting the prompt he wanted to so he’s recorded the MAP file – it’s very easy with Allinea MAP. We’ve seen another one of those. And he sent it to the performance guy who’s going to have a look. We can load it up here. You can lock directly into remote clusters from our remote client now on your MAP recorder. So you don’t actually need to be physically present or [popping?] the files around from user directory and stuff like that at all.
Here we have the report file. Now this is an open MP program. In the 5.0 versions of MAP, we now have a full open MAP support – surely, a lot of people have been waiting for. We’re going to see a bit of that here today. The first thing our intrepid performance analyst sees here is wow, there’s a huge amount of gray up there. Gray is overhead, as time is just being wasted on them.
The dark green color we’re seeing is the main threat computer, and that’s without an new open MAP helping at all. That’s also basically a waste of time. The light green, sort of power, that’s 18% so you can see, “Yeah. This isn’t performing well.” In fact, MAP has already taken exactly to the reason why it’s not performing well down here, most of the time, it’s being spent here in this [swap erase?] line, just popping the data back and forth on the main thread. So he looks at this and he says, “Well, you know, you’re copying the data.” Come on, this is a C program guy. You can just swap the points. We’ll do that instead.
You can just edit it right inside MAP now until you decide [they’re?] out. He doesn’t need to go and find it. Because sometimes, when you just want to do a small fix, you can do it here. We’re not saying, stop using, because is the one true editor, not Emacs. But you can make the changes without having to come out of your zone. It’s about keeping concentrated on what you’re doing. He makes this little change here and saves it. And he can build this as well. There we go. Now, I’m going to close that down and record a new version doing the same thing again, but now we want to see it running with the changes. Off it goes.
It’s running up here along. The nice thing about being able to edit directly at the tool as well, is that if you have an allocation in the cluster, you don’t need to cancel that allocation and come out again and then wait for your new job to be submitted. Because you can do it quickly here and I can do it while the allocation is still running. Of course, this is something you might not like but for you, it’s a very efficient way to work. Now, the problem for performance [on list?] here, “Oh, no. Error validation check failed.” So it’s gone faster, he’s broken it somehow. Now, we don’t need to go back and reason with it, we need to debug out. Happily, there’s one built in.
Switch over to debugger. It’s already set up, configured to repeat exactly what happened before but under the debugger control now. Here we are. It’s the same familiar interface but now, the debugger has taken control of that. And the code he was working on is just up here. Although? there’s a new code. We can just run straight to here. Let’s have a look to see what’s happening. One of the things you might want to do is to visualize these arrays. Maybe if it’s interesting – specifically what’s happening here – you can say, “Okay, I’m going to put a trace point in.” This is something we’ve had in DC for a while. It can just do it in the offline mode. But it’ll just track how those values are changing over time. So if we let that run for a couple of iterations we can have a look at the output here. See, “Oh. They’re the same values.” In fact, what’s happening here is that we’re copying, but we’ve messed the swapping.
You can see, with the the debugger, you’re copying the new values to the values. But then, the old values are not being swapped all over and you’ve got a cycle about those instead of swapping them. You can find that you can fix it, then you can go and profile it again and get your win.
insideHPC: It’s really about enabling this kind of workflow and taking out as much of the distractions as possible.
Mark O’Conner: Right. Just getting it out of the way and let the job get done.
insideHPC: Very nice.