Thanks for the clarification. I agree with all of what you have written/stated. I also have not seen rewards, but I personally think that if I have a big stick for a vendor, I should at least have an equal sized carrot… but I agree that the sticks in the HPC space greatly outnumber the carrots.

Cheers

Sorry about coming off somewhat strong, this is a bit of a hot button for us. We find that the folks who don’t take us seriously try to use us as a blunt instrument against their favorite vendors, wasting our time and resources, in order to get a better deal from their favorite vendor.

I don’t think Shai is saying don’t benchmark. I think he is saying that you have to pay for his software if you want to use it for any purpose. That’s sound business IMO. I would like to see a cluster they maintain with their software accessible for testing, but we may have to bear our own hardware costs for this. He is running a lean business, and hardware isn’t cheap.

We let our customers into our firewall to get access to machines we are building, for their testing. Also we let them in to experiment on machines we are working on, and try to give them a feel for what the performance difference is for what they want.

By letting the customer into the firewall to beat on a machine, we let them show us what tests are important to them. They help us understand what data they are looking for, and then we try to generate it. It also doesn’t cost us anything apart from a little time, and we gain as much as we spend.

Shipping a machine is generally possible. However, as noted, some like to use us as a blunt instrument. Which means we waste money/time/effort shipping the machine, we lose on time for alternative uses of the machine. And if we have invested in building the machine rather than having it part of a committed purchase contract, we have to ask ourselves what the risk of potential return is. For more common and lower cost gear, this isn’t nearly the issue it is with some of the very specialized gear, that would be hard to resell, and is of less use for us for development/testing purposes.

I am all for real benchmarks. Of course, every now and then we get people running ‘benchmarks’ which don’t actually measure what they purport to measure (being say, purely cache bound rather than IO bound) … part of this is working with the people to understand what their real use case is, and testing as close to this as we can, and the other part is to ‘detox’ the FUD/misinformation out there.

More to the point, we are very open about what we test, how we test, etc. More often than not, customers tell us they get the same numbers to some precision that we have measured. If this is the case, then we have done our job well. Because if they get different numbers, then this is a diagnostic event detection, that means something changed, and usually not the way it should.

As for structured contracts, allow me a grimace. I have never … ever … seen a reward for exceeding expectations, and have only seen (very one sided) penalties for missing them. I have never seen a government or research/edu institution accept late fees for paying late. We incur *significant* risk when loaning money like this, and there is no reward we can collect to pay for that excess risk. This has been abused multiple times. So we now review every request for credit (requiring applications from everyone who requests credit) very carefully, and offer alternatives with incentives for taking them. All of this is spelled out in our quotes these days. Its risks like these that eventually drove LNXI under.

If you’d like to talk about this offline, my email is landman _at_ scalableinformatics _dot_ com.

With all due respect, I’m not “beating” up anyone. I’m simply stating that I believe the customer (and btw, my pockets are not empty) should be able to independently verify results. Whether that is through a loaner, a try and buy, access to existing boxes or consultation with existing trusted sources, I care not.

What I do take exception to is the thought that anyone that questions results, no matter how extraordinary they are, should be accosted for suggesting that results need to be independently verified. I’d be perfectly happy to let you (hypothetically) propose benchmarks results, then structure a contract that compensates for missing proposed results (or rewards for that matter, if you surpass).

But what do I know, I just buy, install and run the stuff… I don’t sell anything. I’ve leave that to you smart guys.

So we have taken a different tack with regard to evaluations. We have engineering/demo units in-house that we use, and in some cases, will loan out to customers. We will not pre-configure our largest possible machine, and send it out with a try and buy. And yes, we were asked to do this in the last two weeks. If we have a unit in the lab being built, we can give access to it for a window for tests. And we do. Most customers whom are serious about purchasing units from you have no problem with this. Customers who want to beat up their preferred vendors? They don’t like this so much. Removes the shock-factor when the preferred vendor reps walk through the data center and see your logo sitting where they think theirs should go.

Shai rightly points out that vendors that have been doing this in the past, are being bought for pennies on the dollar at Chapter 11 auctions, or are being forced to sell themselves to others to continue operations. There is a reason why … bad business practices do not make for long term surviving companies.

We (Scalable Informatics) have on our plans to purchase a license soon, and like with our other units, may open this up to specific customer testing. Don’t beat Shai up for running with sound business practices. Do beat up the companies that lend gear out with no hope of recouping the cost of that gear. Those are companies that won’t be around in short order.

That’s pretty much what I expected you to say.

Good Luck with that.

*Customers* (i.e. people/organizations that pay money to use the product) have successfully benchmarked our product and technology many times. It passed many acceptance tests and deployed all over the world (3 digits for the number of different sites).

I will be happy to provide you with some customer benchmark data (send email to Shai at ScaleMP dot com).

So – the answer is that you need to buy it to “benchmark” it. A small-scale deployment costs much less than what a serious prospect would invest in benchmarking (in man-days).

P.S.
ScaleMP does not engage with “folks who are interested in testing-only”, as we have a (successful) business to run, and focus our resources on revenue-generating activities. I am aware of some other companies that are more open with “testing-only” scenarios; unlike me, I guess they enjoy the chapter 11 rollercoaster

You are right – AMD was there first. Unfortunately, in order to be “mainstream” in x86, due to its market share, Intel needs to play as well.

Therefore *Intel* “brings NUMA solutions to the mainstream x86 architecture”. AMD for years offering us all excellent NUMA solutions – and was the first to embrace it. Intel recent adoption of NUMA architecture brings it to mainstream and we can expect for broader ISV acceptance (NUMA awareness, optimizations, etc.).

Re-reading I suppose it might mean that it’s bringing NUMA to *Intels* mainstream products, and if that’s the case I apologise, but it didn’t come across that way.

I’m well aware of what ScaleMP make.

Your comment about QPI has no relevance to the article, which is about building large shared memory systems from COTs servers.

Time to wheel on Linus’s quote from 2004 when Intel announced their AMD compatible 64-bit architecture..

Actually, I’m a bit disgusted at Intel for not even _mentioning_ AMD in their documentation or their releases, so I’d almost be inclined to rename the thing as “AMD64″ just to give credit where credit is due. However,
it’s just not worth the pain and confusion.

Intel’s HyperTransport, er, QPI, is nothing new and I’m really puzzled why he would think it was different..