benchmark results.json

[Jaefromkorea]

Dear Adrian Sampson,

I am testing benchmark with accept compiler follow tutorial.
However, I got some weird result from command make exp
It seems like only sobel works properly, and the other application do not give the result .

could you please explain what i have done wrong?

My purpose of running these application is that

1. i first want to generate the benchmarks with accept compiler in order to get QoR(quality of result)

2. I want to generate the binary file with this and simulate in NoC simulator call sniper in order to test the floating point exchanges between noc.

I got this idea from the below paper

AxNoC: Low-power Approximate
Network-on-Chips using Critical-Path Isolation
Akram Ben Ahmed�, Daichi Fujikiy, Hiroki Matsutani�, Michihiro Koibuchiz, and Hideharu Amano�

I just want to simulate exactly same like the paper to see the more precise result.

Thank you
{
"blackscholes": {
"isolated": {
"desync": [],
"loopperf": [],
"npu": []
},
"main": [],
"stats": {
"desync": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.0003631114959716797,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"loopperf": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.0003218650817871094,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"main": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.6068341732025146,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"npu": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.0003249645233154297,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
}
}
},
"canneal": {
"isolated": {
"desync": [],
"loopperf": [],
"npu": []
},
"main": [],
"stats": {
"desync": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.00036406517028808594,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"loopperf": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.00031495094299316406,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"main": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 29.426498889923096,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"npu": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.0003910064697265625,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
}
}
},
"fluidanimate": {
"isolated": {
"desync": [],
"loopperf": [],
"npu": []
},
"main": [],
"stats": {
"desync": {
"all": 6,
"base": 6,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 5.352258920669556,
"train-bad": 6,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 6
},
"loopperf": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.0003380775451660156,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"main": {
"all": 6,
"base": 6,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 107.3139750957489,
"train-bad": 6,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 6
},
"npu": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.0003781318664550781,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
}
}
},
"sobel": {
"isolated": {
"desync": [],
"loopperf": [
{
"config": "loop at sobel.c:50 @ 1, loop at sobel.c:51 @ 6, loop at sobel.c:56 @ 1",
"error_mu": 0.25624184701956954,
"error_sigma": 0.0,
"speedup_mu": 1.2263029031129764,
"speedup_sigma": 0.049871042276112255
},
{
"config": "loop at sobel.c:56 @ 1",
"error_mu": 0.25624184701956954,
"error_sigma": 0.0,
"speedup_mu": 1.1543336900156391,
"speedup_sigma": 0.01722010757024235
}
],
"npu": []
},
"main": [
{
"config": "loop at sobel.c:50 @ 1, loop at sobel.c:51 @ 6, loop at sobel.c:56 @ 1",
"error_mu": 0.25624184701956954,
"error_sigma": 0.0,
"speedup_mu": 1.2263029031129764,
"speedup_sigma": 0.049871042276112255
},
{
"config": "loop at sobel.c:56 @ 1",
"error_mu": 0.25624184701956954,
"error_sigma": 0.0,
"speedup_mu": 1.1543336900156391,
"speedup_sigma": 0.01722010757024235
}
],
"stats": {
"desync": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.0002830028533935547,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
},
"loopperf": {
"all": 11,
"base": 4,
"composite": 3,
"test-bad": 8,
"test-optimal": 2,
"test-suboptimal": 0,
"time": 5.6406919956207275,
"train-bad": 1,
"train-optimal": 8,
"train-suboptimal": 2,
"tuned": 9
},
"main": {
"all": 11,
"base": 4,
"composite": 3,
"test-bad": 8,
"test-optimal": 2,
"test-suboptimal": 0,
"time": 10.55833101272583,
"train-bad": 1,
"train-optimal": 8,
"train-suboptimal": 2,
"tuned": 9
},
"npu": {
"all": 0,
"base": 0,
"composite": 0,
"test-bad": 0,
"test-optimal": 0,
"test-suboptimal": 0,
"time": 0.00033092498779296875,
"train-bad": 0,
"train-optimal": 0,
"train-suboptimal": 0,
"tuned": 0
}
}
}
}

[sampsyo]

Hi! Can you include the verbose output of a tool run for a benchmark that produces zero results?

[Jaefromkorea]

I have used same code from accept-app benchmark from your code

…

On 2 Apr 2019, at 17:21, Adrian Sampson ***@***.***> wrote: Hi! Can you include the verbose output of a tool run for a benchmark that produces zero results? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[sampsyo]

Yes, I know! But can you include the verbose output from a tool run for one of those benchmarks?

[Jaefromkorea]

I don't know how it should look like the verbose output but from the tutorial I saw the command accept -kfv build. the result of the this command for blackscholes is the zip file i have attached and from accept -v run command i got the file txt file in saved_output file [image: image.png] one thing more I don't understand is that the application blackscholes delete the output file after the computing [image: image.png] I always have to find the result at tmp file with flag command -k Please tell me if i am doing wrong thank you 2019년 4월 2일 (화) 오후 8:21, Adrian Sampson <notifications@github.com>님이 작성:

Yes, I know! But can you include the verbose output from a tool run for one of those benchmarks? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#53 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZN5DNKmv8qdMdY-754_3swsyNZ5pDfDks5vc5-vgaJpZM4cYUJ-> .

65536 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 0.000000000000000000 very long 0s ...

[sampsyo]

Hello! It looks like the image attachments didn't work. Can you paste the actual, text output?

[Jaefromkorea]

hi, *jae@jae-VirtualBox:~/accept-apps/blackscholes$ accept run -v* precise output: /home/jae/accept/saved_outputs/HXm4P4L1gB10eSrn89iG.txt 0 optimal, 0 suboptimal, 0 bad suboptimal configs: bad configs: *jae@jae-VirtualBox:~/accept-apps/blackscholes$ accept -v run* executing setup target starting baseline execution for training evaluating 0 base configurations evaluating tuned parameters evaluating combined configs 0 composite configs 0 optimal, 0 suboptimal, 0 bad *jae@jae-VirtualBox:~/accept-apps/blackscholes$ accept -kfv build* building in directory: /tmp/tmpYhyWCf /home/jae/accept/build/built/bin/llc -O2 blackscholes.orig.bc > blackscholes.orig.s /home/jae/accept/build/built/bin/clang++ -lm -pthread -o blackscholes.orig blackscholes.orig.s rm blackscholes.orig.s 2019년 4월 3일 (수) 오후 7:25, Adrian Sampson <notifications@github.com>님이 작성:

…

Hello! It looks like the image attachments didn't work. Can you paste the actual, text output? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#53 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZN5DDEmNeWo4e0_fuM7fePS-ZlxRURtks5vdOQXgaJpZM4cYUJ-> .

[Jaefromkorea]

this zip file is the all the result from blackscholes after the build 2019년 4월 3일 (수) 오후 8:46, jaechul lee <twims0@gmail.com>님이 작성:

…

hi, ***@***.***:~/accept-apps/blackscholes$ accept run -v* precise output: /home/jae/accept/saved_outputs/HXm4P4L1gB10eSrn89iG.txt 0 optimal, 0 suboptimal, 0 bad suboptimal configs: bad configs: ***@***.***:~/accept-apps/blackscholes$ accept -v run* executing setup target starting baseline execution for training evaluating 0 base configurations evaluating tuned parameters evaluating combined configs 0 composite configs 0 optimal, 0 suboptimal, 0 bad ***@***.***:~/accept-apps/blackscholes$ accept -kfv build* building in directory: /tmp/tmpYhyWCf /home/jae/accept/build/built/bin/llc -O2 blackscholes.orig.bc > blackscholes.orig.s /home/jae/accept/build/built/bin/clang++ -lm -pthread -o blackscholes.orig blackscholes.orig.s rm blackscholes.orig.s 2019년 4월 3일 (수) 오후 7:25, Adrian Sampson ***@***.***>님이 작성: > Hello! It looks like the image attachments didn't work. Can you paste the > actual, text output? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#53 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AZN5DDEmNeWo4e0_fuM7fePS-ZlxRURtks5vdOQXgaJpZM4cYUJ-> > . >

[sampsyo]

Hmm… zero configurations? I'm not sure exactly what's causing that! I'm really going to have to depend on you to do the debugging yourself, though. Can you look through the code to see where configurations are generated and trace backward to see why none are being found for you?

(Your attachments still aren't working. Please look at the GitHub thread.)