Should I run my regression testing programs on AMD and Intel chips?

Question

Now I plan to test 32-bit, 64-bit, Windows XP Home, Windows XP Pro, Windows Vista Home Basic, Windows Vista Ultimate, Windows 7 Home Basic and Windows 7 Ultimate ... all with the latest service pack.

However, now I'm wondering if it is worth testing on both AMD and Intel for all of the scenarios listed above, or is it a waste of time?

Note. This is a security application for ordinary users.

Answer 1

I feel that it would be advisable if you had a lot of built-in assembler language with manual encoding or some incredibly tight timings (which you are not going to meet with this choice of OS anyway).

If you use ready-made commercial compilers, then you can be sure that they are going to generate code that runs on all normal processors.

Of course, no one could prove that they did not need to test on a specific platform, but I think there are more reasons for the difference in the platform than worrying about the processor (all different multi-core / hyper-threaded permutations, for example, which can cause all your multi-threaded errors with code differently)

Answer 2

Only if you program in the assembly and use the extended, specific sets of developer commands. But since AMD and Intel have cross-licensing agreements, this is more a historical issue than the current one.

In each other case (for example, using a high-level language), it is the work of compiler writers to ensure that the code complies with x86 requirements and runs on each CPU.

Oh, and besides FDIV Error, CPU vendors usually make no mistakes.

Answer 3

I think that you are looking in the wrong direction for testing scripts.

Yes, it is possible that your code will work on Intel, but not on AMD, or in Windows Vista Home, but not in Windows Vista Professional. But if you do not do something very closely related to low-level programming in the first case or to the details of the OS implementation in the second, the chances are small. You could say that it’s never scary to experience every conceivable scenario. But in real life there should be a certain limit of resources available to you for testing. Testing on different processors or different OS in most cases does not test your program, it tests the compiler, OS or processor. How much time do you have to check other people's work? I think your time will be better spent testing more scripts in your own code. You do not give details about what your application does, but just to take one of my own examples, it would be much more productive to conduct a daily test for selling products that our own company produces in comparison with products that we resell from other manufacturers, or testing sales tax rules for different conditions or something else.

In practice, I rarely test deployment on Windows and deployment on Linux, not to mention different versions of Windows, and I rarely burn on it.

If I were to write low-level device drivers or some, this would be a different story. But ordinary applications? Do not waste your time.

Answer 4

Of course, it sounds like this would be a waste of time for me - in what language (s) are your programs written?

Answer 5

I would say no. If you are not writing your application in assembler, you should be far enough away from the processor so as not to worry about the differences. Processors will support Windows, the API of which you interact with (depending on the language). If you are using .NET, ONLY for the foreseeable problem you will get is if you are using a version of the framework that these platforms do not support. Given that they are all XP or later, you should be fine. If you want to worry, make sure your application works well with Vista and the later security model.

Answer 6

The question is probably "what are you testing." It is unlikely that any of the tests would test something that would potentially be different between the AMD and Intel hardware platforms. Differences could be expected at the driver level, but you don't seem to be testing your software for every existing bit of PC hardware. Most likely, there will be far more differences between the different levels of the Windows Service Pack than between the AMD and Intel processors.

Answer 7

I believe that there are some features in your code that (whether you know it or not) take advantage of some processing / optimization in a given situation that can seriously affect the result. Maybe a keyword.

I would say that in general you are unlikely to have to worry about this. If you do this on multiple machines anyway, mix them with them. But I would not talk about it.

Answer 8

I would never have completed all of my regression tests for both AMD and Intel, if I had not specifically fixed a problem that is unique to one of them. This is what regression testing is.

Testing on the other hand ... I would not expect any difference. So, again, I would not discuss unit tests on both until I saw a problem specific to AMD or Intel.

Answer 9

If you rely on accurate / consistent floating point results, then yes, definitely.