[mvapich-discuss] mvapich-discuss messages bounce: Too large headers
Gus Correa
gus at ldeo.columbia.edu
Tue Aug 1 13:16:13 EDT 2017
Thank you so much, Dr. Panda.
I appreciate your help.
I wonder if the large message
headers affected also other list subscribers.
Anyway, hopefully it will be a simple fix, something in the
mailing list server configuration.
FYI, our local sys. admin. (Cc'd),
just told me he had increased the max. header size even more, to 64kB.
However, I still don't get mvapich-discuss messages.
Regards,
Gus Correa
On 08/01/2017 01:04 PM, Panda, Dhabaleswar wrote:
> Hi Gus,
>
> Thanks for your note. Sorry to know about this issue. The IT infrastructure (including mailing lists) within OSU have been going through some unification/reorganization across various departments. It could be a side-effect of this. Let me contact the appropriate people about this issue and get back to you.
>
> Thanks in advance for your patience.
>
> DK
> ________________________________________
> From: Gus Correa [gus at ldeo.columbia.edu]
> Sent: Tuesday, August 01, 2017 12:31 PM
> To: mvapich-discuss at cse.ohio-state.edu
> Cc: Subramoni, Hari; Panda, Dhabaleswar; Jonathan Sattleberger; Gus Correa
> Subject: mvapich-discuss messages bounce: Too large headers
>
> Hi Mvapich-Discuss list adminstrators
>
> I have not received the messages from mvapich-discuss mailing list
> for several weeks now.
> I have also been removed from the list twice,
> because the messages bounce on our mail server,
> back to your mail server.
> I have reinstated myself to the list twice since then,
> but that didn't solve the problem.
>
> Our local email administrator (Cc'd here)
> has whitelisted mvapich-discuss
> (it had been blacklisted due to message bouncing)
> and increased the maximum incoming message header size to 32kB,
> which is well above the typical/median header size (8-16kB)
> However, the mvapich-dicuss messages have enormous headers,
> above 32kB, and continue to bounce.
>
> FYI, I enclose an example of a message that was accepted,
> with a big header, and lots of Microsoft Exchange diagnostics.
> However, please note that this was still a "small" header message,
> and mvapich-discuss messages normally have even bigger headers.
>
> Could you please take a look at what is going on, and hopefully
> reduce the overhead in the message header size?
>
> I have been a user of mvapich and a list subscriber for a while,
> and would love to be able to continue to read the postings.
> I know this may not be the right forum for this request,
> but I couldn't find any other pathway to reach you out.
> I won't probably even receive this message back as a list subscriber,
> so please feel free to email me directly, if you want.
>
> Thank you for your attention and help.
> Gus Correa
>
> ****************************************************************
> Below is an mvapich-discuss message with a "small" header.
> The norm is much more!
> ****************************************************************
>
>
>
>
> -------- Forwarded Message --------
> Return-Path: <mvapich-discuss-bounces at mailman.cse.ohio-state.edu>
> Received: from lamont.ldeo.columbia.edu (lamont.ldeo.columbia.edu
> [129.236.10.30]) by astolat.ldeo.columbia.edu
> (8.14.7/8.14.7/ASTOLAT-LDEO-2.1-d2110) with ESMTP id v6BLcZKI065143
> (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-GCM-SHA256 bits=128
> verify=OK) for <gus at ldeo.columbia.edu>; Tue, 11 Jul 2017 17:38:35 -0400
> Received: from sophos1.ldeo.columbia.edu (sophos1.ldgo.columbia.edu
> [129.236.19.120]) by lamont.ldeo.columbia.edu
> (8.15.1/8.15.1/LDEO-2.1.1-uw2k7f) with ESMTP id v6BLcXmp010567 for
> <gus at ldeo.columbia.edu>; Tue, 11 Jul 2017 17:38:33 -0400 (EDT)
> Received: from sophos1.ldeo.columbia.edu (localhost.localdomain
> [127.0.0.1]) by localhost (Email Security Appliance) with SMTP id
> 191B93027D8C_9654559B for <gus at ldeo.columbia.edu>; Tue, 11 Jul 2017
> 21:38:33 +0000 (GMT)
> Received: from NAM03-CO1-obe.outbound.protection.outlook.com
> (mail-co1nam03on0139.outbound.protection.outlook.com [104.47.40.139]) by
> sophos1.ldeo.columbia.edu (Sophos Email Appliance) with ESMTP id
> 79CC93015C81_9654557F for <gus at ldeo.columbia.edu>; Tue, 11 Jul 2017
> 21:38:31 +0000 (GMT)
> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
> d=buckeyemailosu.onmicrosoft.com;
> s=selector1-mailman-cse-ohiostate-edu04b;
> h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
> bh=r39czZCZIAsqAMII0hXZgCeDeCIJQoelrkGOdobVSJ4=;
> b=D9oV0M5BOw08FzBjNGKNHgKnGPwJNS+M03Gc0MAWNwvX4wKzZJ9jGaYUG5SMUsS7z5LdD8DL92U+0oCCKGxeCXNvu97aqzuX8ynMvhGJjT+9+/oguYOvVol8950/bxXtR4sqUzO2XxD9h/BTFtWakDlu84AkVysxRFD6LPAIWL8=
> Received: from CY4PR01CA0021.prod.exchangelabs.com (10.169.249.31) by
> CY1PR0101MB0923.prod.exchangelabs.com (10.160.139.26) with Microsoft
> SMTP Server (version=TLS1_2,
> cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1240.13; Tue,
> 11 Jul 2017 21:38:29 +0000
> Received: from BY2NAM05FT042.eop-nam05.prod.protection.outlook.com
> (2a01:111:f400:7e52::204) by CY4PR01CA0021.outlook.office365.com
> (2603:10b6:903:1f::31) with Microsoft SMTP Server (version=TLS1_2,
> cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1261.13 via
> Frontend Transport; Tue, 11 Jul 2017 21:38:28 +0000
> Authentication-Results: spf=none (sender IP is 128.146.138.10)
> smtp.mailfrom=mailman.cse.ohio-state.edu; accre.vanderbilt.edu;
> dkim=none (message not signed) header.d=none;accre.vanderbilt.edu;
> dmarc=fail action=none header.from=osu.edu;
> Received-SPF: None (protection.outlook.com: mailman.cse.ohio-state.edu
> does not designate permitted sender hosts)
> Received: from cio-socc-esr05.osuad.osu.edu (128.146.138.10) by
> BY2NAM05FT042.mail.protection.outlook.com (10.152.100.179) with
> Microsoft SMTP Server (version=TLS1_2,
> cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1240.9 via
> Frontend Transport; Tue, 11 Jul 2017 21:38:05 +0000
> Received: from CIO-TNC-HT05.osuad.osu.edu (localhost [127.0.0.1]) (using
> TLSv1.2 with cipher AES256-SHA256 (256/256 bits)) (No client certificate
> requested) by cio-socc-esr05.osuad.osu.edu (Postfix) with ESMTPS id
> ABDDB96; Tue, 11 Jul 2017 17:38:04 -0400 (EDT)
> Received: from mailman.cse.ohio-state.edu (164.107.115.39) by
> smtp.service.osu.edu (164.107.81.254) with Microsoft SMTP Server (TLS)
> id 14.3.319.2; Tue, 11 Jul 2017 17:38:04 -0400
> Received: from mailman.cse.ohio-state.edu (localhost.localdomain
> [127.0.0.1]) by mailman.cse.ohio-state.edu (8.14.4/8.14.4) with ESMTP id
> v6BLc1Gx027660; Tue, 11 Jul 2017 17:38:01 -0400
> Received: from CIO-KRC-HT01.osuad.osu.edu (cio-krc-ht01.osuad.osu.edu
> [164.107.81.37]) by mailman.cse.ohio-state.edu (8.14.4/8.14.4) with
> ESMTP id v6BLc0vh027657 for
> <mvapich-discuss at mailman.cse.ohio-state.edu>; Tue, 11 Jul 2017 17:38:00
> -0400
> Received: from [164.107.94.24] (164.107.94.24) by smtp.service.osu.edu
> (164.107.81.254) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 11
> Jul 2017 17:38:00 -0400
> MIME-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
> From: Xiaoyi Lu <lu.932 at osu.edu>
> In-Reply-To: <86c08744-2611-c473-3673-c2cee6569a3c at os.inf.tu-dresden.de>
> Date: Tue, 11 Jul 2017 17:38:00 -0400
> Message-ID: <6F5B866C-76BD-46AF-958B-215796EC680B at osu.edu>
> References: <6e57fd7f-e3d5-b25f-165e-666e74bf57cf at os.inf.tu-dresden.de>
> <60845F2C-CFCD-4E84-8833-40DFB6A995AF at osu.edu>
> <4430d147-3bf1-076b-f937-13f8f39d5784 at os.inf.tu-dresden.de>
> <CAA8C3E8-73B0-4D15-9725-D1D68AC3BCBE at osu.edu>
> <cd1954f7-7075-85a2-e2bd-7b1d35f63d6f at os.inf.tu-dresden.de>
> <A4DFC7F5-DC92-494E-9A9E-B95C16414D02 at osu.edu>
> <86c08744-2611-c473-3673-c2cee6569a3c at os.inf.tu-dresden.de>
> To: Maksym Planeta <mplaneta at os.inf.tu-dresden.de>
> X-Mailer: Apple Mail (2.2104)
> X-Originating-IP: [164.107.94.24]
> x-header-sapphire: true
> X-MIME-Autoconverted: from quoted-printable to 8bit by
> mailman.cse.ohio-state.edu id v6BLc0vh027657
> Subject: Re: [mvapich-discuss] Bad performance of IVSHMEM with OSU
> benchmarks
> X-BeenThere: mvapich-discuss at cse.ohio-state.edu
> X-Mailman-Version: 2.1.20
> Precedence: list
> List-Id: Discussion about MVAPICH and MVAPICH2 software
> <mvapich-discuss.cse.ohio-state.edu>
> List-Unsubscribe:
> <http://mailman.cse.ohio-state.edu/mailman/options/mvapich-discuss>,
> <mailto:mvapich-discuss-request at cse.ohio-state.edu?subject=unsubscribe>
> List-Archive: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/>
> List-Post: <mailto:mvapich-discuss at cse.ohio-state.edu>
> List-Help: <mailto:mvapich-discuss-request at cse.ohio-state.edu?subject=help>
> List-Subscribe:
> <http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss>,
> <mailto:mvapich-discuss-request at cse.ohio-state.edu?subject=subscribe>
> CC: MVAPICH Discuss List <mvapich-discuss at cse.ohio-state.edu>
> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: 7bit
> Errors-To: mvapich-discuss-bounces at cse.ohio-state.edu
> Sender: mvapich-discuss <mvapich-discuss-bounces at cse.ohio-state.edu>
> X-CFilter-Loop: Reflected
> X-EOPAttributedMessage: 0
> X-Forefront-Antispam-Report:
> CIP:128.146.138.10;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10019020)(6009001)(39840400002)(39450400003)(39860400002)(39410400002)(39850400002)(39400400002)(2980300002)(428002)(377454003)(377424004)(199003)(176002)(156002)(24454002)(189002)(50986999)(47976999)(6706004)(76786999)(76176999)(7846003)(76806002)(57306001)(6306002)(7416002)(6392003)(8656002)(356003)(33656002)(46406003)(7736002)(36756003)(47776003)(8666007)(53546010)(93886004)(66066001)(50466002)(109096001)(97756001)(626005)(7406005)(19810500001)(1720100001)(7596002)(2950100002)(42882006)(105586002)(106466001)(78456001)(101416001)(305945005)(5890100001)(189998001)(5660300001)(966005)(4326008)(478600001)(229853002)(38730400002)(107886003)(16670700002)(50226002)(110136004)(230700001)(83716003)(146002)(9036002)(6246003)(82746002)(88552002)(75432002)(8936002)(6116002)(2906002)(6862004)(8676002)(39060400002)(3846002)(23726003)(3940600001)(473944003);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR0101MB0923;H:cio-socc-esr05.osuad.osu.edu;FPR:;SPF:None;MLV:ovrnspm;A:1;MX:1;PTR:cio-socc-esr05.osuad.osu.edu;LANG:en;
> X-Microsoft-Exchange-Diagnostics:
> 1;BY2NAM05FT042;1:JterWTZRGOqsABu78FkumWJpLo6jJJaTuIvhLq+t85mR5yrzggiHzArkA88OXFE9q90gkkv9BICrXlR4ewBN+XIXSpn3KCinKN9rn/xWWakq/1tKuN+mSbS3MM6Wl201fI74JTU2B+9060tjyeDaa1qiP/+w9tfLuWVot1tno1vYDHYn0h3dsmmBo0s9UgVKzMwlkpa3yThFGjpmRkUgOoZHBMV9viscNB/hq+frCb6tw/N7l7jba/UQDmWExWL4QLIoNNuYFRwlhp1Ucamzrxb/XQ9o3URqwAjGxpbJSw7Ef+l1VRpFGxD5fLTqdT1Vl8r42jlBh5v4fuiBpnIMQTzVyeLePKR7C7WiuYU7wYCbZ+uT766M4yRyp2giX4O0QQo78XSPVbG26pNgiVOpgrGpcdaA90oIC4fuRM0YwE7B4XJqRiUFHzVwbXGD2z6yfkLXe9jRRO6HuEtxpW/3ub9byUfTdKYNXLLwMivEcr0GLlFX5W2YQ5cJCHpANP9ewEIlPCsaJovSMxztLiUtWK8MphESgjTmsvSYNf7GIjAJpJ6y0Zf+Y2eRslzITdbZKsMyaYJ6Zc/m96sK7wCzLcE39UolrJJjjGF5WbzVFjLoVbzHrNUV1urie6U0kGYrhafAYEDA3GZg4quRQ0Lgg63q9QPLLXNBx8n0qp5yK7OMO/z23aQmi7ylWiT9MvIwq6OmrehEqDqY1PswFGxbK06Egjf1MbNDg0liFLG84mJXlAU6G1N0JBWCS9y+FUOt0xnFyiJu5NBpDuUeRDdr5gaHKaE2Vun2b8i/qwlJ1P7mpi9XZc0ynGvJaiaaA4oF3SqoPS8C3ZrZAalDqA9CZjlKZyhyaipgm29D7uNm7EL8lik7M7the5co94toBXlKaNElTmk2LtNie6JHxyBLk/gyijVE7ujQsVXmA6F52D
> CCLevO7ScEwEswWpPvAW8s
> X-MS-PublicTrafficType: Email
> X-MS-Office365-Filtering-Correlation-Id:
> 55b3ef0c-bff6-47dc-1b69-08d4c8a52ae9
> X-Microsoft-Antispam:
> UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(421252002)(2017030254075)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:CY1PR0101MB0923;
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;3:axGg2Yku1lpWx6FzkA715hqzQThVq4oZEwWQmIq6YZ3lqTGEDC/6lFvLJtkJaHudPrvSU/+LEzhCRPxPaeim7ns2/D0NnbuYrMtR8gYEPAAXmMH17gVLAIvJm2Ud7ntRzaubsdbnepqYrHx+YSJyxXg+9Ib6SvgKhpKLmDZN6Mb2KhrJS1SxXedpfJlrWw1/Oz5zfuHsGWUny69gCsHAhXgr6uXZJP4eLHiRtzzxRa+2J2KvUk8Dp4geXXSxVcLcuUlijiAJMZmXm4NqtyXGI5EDCAijaYogL0DNh/7/Wpdg24se4s+txiRCKj2lhUVMOlcj1ImetTch4fy5pu/BTsihcSyBOZPYPTAlTKcAlB2oJ3dXxiUn6mNpOOzkcFMm51vF3E/K1Ct/HlGgEe07TYqfr/KVYn6FD9IIkplLiZ2Hn6pFFHg+kx2jh77Fv6t+CeA3um8K2U+DTosF76iG4gJZLu3LYewH5YpafrVafxrO/Vp3e7AaDYQf5BX9cIgJ2PkcApMQncP08gIGfX8fVW/P0ZiN1cnweI1l2OBD5g3ZANzVBnVxgP+uslSpcKQnpQz5JyP04G/U8nzDb56U9R16/SRyoFo2JC+YOB29xnqqn4odAlnFDZYc4M+R5MMNdfrHCtH7X07N5rw6WkdR3+N6IzLOaL7d7e6S1PQ6aAblgnnbkIAf1f9rd/RM88UKE/LdhfnWe8hsK1gyTjOHeunQOavyzdHRItRilz84faPfQsTw/yoFKr9Or/UpD6RPdOIZ5aHqxa2craX3y5FmIuaytwEroEwTuqchns4dhQZzBU18lJTUbuqVPGk6WoKGCEUAupLCUeeeJk9ax2i6Y/KrqEeWGvypE5XZYFzTryVc6rv0rBBjVmSS8a9aBO+znceQDbtKpn+AOCtbUU1OghElXRLN7sxXqgbRnxuZyEw=
> X-MS-TrafficTypeDiagnostic: CY1PR0101MB0923:
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;25:XRKjF9OZLwEiV2GzZYEIW6GG/3gL6/mK9JHYl33APsJgvfSjKpcTLDvJ67l7xAieDlkaA1ZO5cZFR3w6qrH4jLBAY8iz65W7hzlR6AYZ/LMEkGQvWp+/u3JRvtUhFkXbt3MXuodayZJASJ/tV6BhpA72DULFUJMNcMJzp8LOPeJ2JpXZJBqJYtrnUJ1Y9jPtb0rJ0yIB/jUJWE6A3TsE95zkNUfBhXfLgfCPgi5Iyz6RgA3gdMSGkW714vucXje3reFUUqudOwDUrsP/CYKyLyobutCF0dqmFlRIWZgL4yd3NOqV2u2bbAyuxbW8AW/fuim8mh3GpWf+cgomauQFL/RYtv1Rddk0Sbo0Zqm9gKXZbuQWIVCCRESGLEgCgOSSW3Va14RRGhBnW9CAY2czAAjeOY2r6vZoSjoslVKcLoaJyxZAOVM1JRRjUnbNZdHO+kGr8TL5Pn+6y/KcWmQ/SW372znPh46VKf7k3mMEQZsydDgqRDPsJnpzcxDyj81tZ8MpWVc7E4tquI/YL4hSkuApNtWyCN8NJAQHCf3Mp2mDNhrAeXkRuYPFSRNUNv7X8kJGRVv0pWeXQ+sPQ8kHHTqp6R8m6TmFKm8gKGsNz7kPOe/XzKYk4tX2cKBCxQ+uy0KGqLBaDiBZHXFimcsFcxN+uHs/MpRTNx1nkxIfTOs0rIvNAgIiqZCWfeynzIRcJY0kqSYdOdIHKpt4D/8j81CJJyO/ip3jmf8e+nmNKp8HGZY3kHhQvLN2adpXk5MJyFmd+F3aud3pyVo2N8oKyOw1otLhEItF1jgN6tNqtDKW+yZfQFPzxcFp0IAH4lW87LFpp7wT16QRB9XVFXWkh1/CStfNQqkVHr0j9uwVJdq1s675pBaWN1fFFW54WLXzStfW63W5wKy/3Fx4/PgPnEBzIdbsFpvm1o2ZZbvI2ME=
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;31:bdVTPJyoyEDr6kI1Nh+OHHrmA6Kwnjo3Ngg+Va5Qwvm06SOeDPGNQKZBz+KcoQAPESb5OsmvPqfCKOMB4Pf0dlZwxJziRU9GgnPlIEsPBIUKDqDdOTFHPJU2AifCG5oyHz8qucyow8MrBCXWz4EGBk6Nska31xJFvmIzZXONk43grxM6+y6cpmFF0/zcOoKpY3UUvJTKDWB1n4XydfAk7D/x4O32+Cb2ArguGnZ0yQaYnc70w+Jboetit0gKFPqrsNVlFr2fsbUidX6K8mkB7b8FM2VyIu0gT2Jsi+L1+Z4tRZsTecn6K09vuUw/w9UsFZPwxnHoeOviJv+QOWQiqRXVDLcTwMV941dxXUDic2w26mu2CdjfEBGZ7hXdfY6DCxPxjXcmo/RWIQrBYG9zO0N8Ecf52AsQvBJY2OpUSidrbFHK6w657WVqSGOKP0LynbZhJb/e5meRukwo1SAR7cJ5XLG8c93Gbv8doPjjjav3AFIzVCG2mnMAP7uJXHfZ7CCRv7ZMbZLXYF7e2nvT4f+1Xp7bYBrUYoTxEzkKm5T3bWcREm4/vCv5v/BgcwhL263jmzDUcoYAVXvMTyIp4EDcWYbBeIEr9+IPnBk2SFA1nWQXk07pr5jSBaCqcFiAGEazr22bt109QJpZ+jOLQ7lrfIddU03ueFu4UHKpsS03rry19GPXWldt+eF9qouvdF7XRkwwpzFYoDBWWIs3Kw==
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;20:2l8FDKOCt7UOzrMu13r+rBKwcyeAGKLkp+pfQZbebiOe1SdJrikES8scMFM49TexH78XcDGypuew+6+/ChyOR3XB6BLrq7cZQdbOFBmaQ1zbyXA5OajZB3r5z775pPkjPmRNFVfQwgXAbzmMw+4zFGqYY3+cHIRG2rL27+PAgdG05qWK7pc7CcOLVAlVIC48jyNC3O2Y9f6vMcVhguAN4ZjMqOwX7bsO+syMGd2BEwp/j40iompIK99Nb3Yp8lviL1iVPfu+/Qj/I4Ub9bHz5BuXbaTcMP/6MU+UG8Ng0swx2+8vIjkrjGN5cAVDn+ZABP+Zwxfa0UC7KIydyDf1ht5duj0LgCgOCQX1MQly2qNOmGsWRk5Z/AtQotyHtzxPPDlKS+bWkNDV/mUxsiVnaBTEaRWqAFwWwfbKE1CY9ZF12GhdJk0iwRogv6gT9wVQnxAJl/h7FnA7/88d2ZtUXKGPrwUnqZEYtIX7MkmKabpJpN5+TRCrhe5NgSzCxj81
> X-Microsoft-Antispam-PRVS:
> <CY1PR0101MB09238FF82C6EC5A5A674DD2C86AE0 at CY1PR0101MB0923.prod.exchangelabs.com>
> X-Exchange-Antispam-Report-Test:
> UriScan:(236129657087228)(48057245064654)(247924648384137)(17755550239193);
> X-Exchange-Antispam-Report-CFA-Test:
> BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(13018025)(5005006)(13016025)(8121501046)(100000703101)(100105400095)(93006095)(93003095)(10201501046)(3002001)(6041248)(20161123562025)(20161123560025)(20161123555025)(20161123564025)(20161123558100)(201703131423075)(201702281528075)(201702281529075)(201703061421075)(201703061406153)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:CY1PR0101MB0923;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CY1PR0101MB0923;
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;4:893EwPFnxtHDrHmjkdm6B5F4c7PAM/oYfIiXQkhlPrun7nv+K+mDJvTXDvyqmvICWYbelE9Pi4DU8BHU87+GZu05whv3ju6bSAZwfkEY9fnkjsJj8U1tTg0lSdqPvQlklEVzuvmBwwu2JwEA6pXuiUsnevFcZVr37WU9e4AKnpwp31ICPAX6/cAqCnSJ7lxU5LQph3leTxJzPl7Iy23OqQ3qvWVDvZvoc2RXAOnKrFEicr1tldTpJY78bmjU3cy0VvdUR3tJlQJAjyJAIoiGbEMh4d5AugganY1dl4PYO4ega3pqsFMxH5z8V/AlwE+m0trnu3IooTWuHebqsChwAQyG605RxKC1+/Zg3TNuhvcZKt2ykf6JgJWkGjefpKfrGzLFrEyQZWkRCyYdB+IOtjQldIb0tf0bAgQ8/xpuWTsBNtjZn3qy9TqsTdPhfq8SNYvuiIhdiYwVGyQwi9sBZZEbYbQgbepQDxtIBkF8tdfXHz9AvifjdeLoMKokElxYaInB+FH4Xo+4pAtl6HQITZm6Vf3mn9IUJiqqMEF0QFZrQwSk+ngx3HUUCi0AGr4Q/8Wkm+6iTWnQ4cIfFe7QouCHaKwNw9nSgfL99rZ4WTAtX0x43PI2XiMUUUKvS+O9mNic13HeG6XpwOD1y6g/62ZjejHKICEpha7bpNmln2TmUH8sEWEPvNgjAFamMgxckyn0ihF02YJBfZU5Z0TUY1q2S1mStZeG+DSjfWs9zz23TJzU9S9BSFHynZazYZzduGH8TKKxFO6mE1As0zYQ0hZ+c1C9KSzVlvoWaTXVar2Ck/Fs965bk1LRdpwH9iYUwwuSaO9b8SxUEa4Mstb7GqDbs0AmJy2+qw6qJ6fR9v0aERuWegzBgfe/nTwRgrUALgR0tZFW2qoWgMf3YqVxLdcylold06ycv7UGBDqDZ6JdrSYp+0g3Uqf/meclHc9pJ49WyQ9CmgC2MsD3wwrEkvZP3hPgc1si0Z6OYmiPpSZDW8otJh3UtROlwg+DgriSTJ2GMMrLhCSXnblMgxW9V8JjrQqpzEG0pRi6cZATg5BtPesqBwvbEBDNn1nwHe1Sn3i5Wa+ISap74n++I7AhzeU0b+kDnJVg+741DkxhSxHPQJirLSshCSj0bkwafCg2HMKeRAnT6qmgIU7rJYmQ8Vkufc/gpj0qt5ZNL9duJjtTfSwJftlDy1gHuicoNpxQv/6x86H1raEWF/DHeky7AYPRFyRNm+PWLHCGpe7f4k4s0muakDB8TwgS9LAkyRHgis8IRR1WqnQVkbwpdy7F3hbORxYYyYq8sAxq1jOveuRyrSrWg9ztML0C5gQPDXUV3yAqfXmemkZrbNJnhJ8aWg==
> X-Forefront-PRVS: 0365C0E14B
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;23:z5go3fKfCOH3L3ca9aTQgQyh5MUHHajDrQb0b6TKiTxriNN6/N//m400hHESmNznzdaSTF+OH1A2bSTNG2EhNF6BosRDOWS1DgEcqdYiv508lFFy9KS+57zo7TU/+2ABS56OSHrmlwb1niis0pBhu0W8it1/lr0r5XZ+YNMkDxuAIeIfwVCRJRicN8W8YXi0L1ZJT/vnPqXdbUBVekRDD0kzyZtfoz2LUHVr9Yg3nWOBvImqRMt31MNsKffUCanKxKf66q+q2aBRYEpcbPcq+mYFG4+YRbClCx9ZAT19rkBPAxGPa3CqkQADe6gXegDJQ/ZvGKba+6xlRsGl6HskSY0jvaGi82j4F9WMRtU4NoWDGOAR3FAM08xaV+ivrHQTZftVolPqokrtnMdyx4Yd2tUbLDCqw5Xwtw3isw1MKh207xo7nk4UsDwNi0YmTUfNJeOkmbgC3d1mVem1jmvlVqYX/QXqAECe4QIxXCQVFIbav+yLtrM2vejtCkX4NZGoD4F0HKQh55TciAMjyUij1LIPBBx3J1U7MLUBpxMy1vvvW5NAk0BoMsgU1rB8IO6Nqg5QPtKM3q/GKE2XX7xJAQzc5Ue+BvGSSqtm9CG5lPMo2WqjUQ4AmjGeMw9i8Y9GJaz3h1SfeYtar3VD6zGZ93Sdq8ci79YPLFHVobxJh9d10qQmNkZ9Dvman8L/A7h7X6l2LUWVRy7drn3eiYB+/yBA5K/3E8fzpuiJPTS2Z9Qn+9sll0q/CfLVSSiqtvzv3I5Bc05qC7XTgx6zro9NKBCuCS9/BGboZ0HzRwpzFLn9H8aP79C1junBV0ruF1wyxYNti/i2/CZMJrwAdY+OC08KVpxsPJl9tHbFdQX6kNWa6tKsfEneCNNYDRGc3ZD8l2hl5+uB0bS+IBkp+Oa02E6PuBW7BmUHiXzT7kvFBsYR/OLaOCMkEqXksViz8/cZVJkXoT2qxETXAA02ZzOwNN9njx4FiSvG9GFGxbPLpvh0jpZIS4Hh3y0+04fq4HO1P0tWMJOHGLqFFQtWYTdo8OoIQEUQPCwzDEIDwHlPB181xFM+KCOgWs0dzbHoMbHiE+BHfLL0Auuy9cRtEsbNjjY0qbqCdNBRhJXSe8agUsb9yVbCfX+AnzSJhAReBct7IMyv3fYy/8QHnqypvA6IxiqFF6Ud7cBgnC5gLY+AFNwfKHp0wY9MgV0nlgpw6tnAzeG84pFTi1IqpZ7u030/wmvOPs2H/cIlnJWORsk+QGHAU+8BQtP941tUfDpYnysUA6qh2Py/FwgZxBFvr4GAMEy0WOhUR5BJmtAplYXtJUU8s/GZTc45alamWXHsg0/t4DlmeBY1HueOVqkHPgFduPczr4Grpnb4uz+JRjEDUjZqL8pyR64fByNR+s/f/GDe6nao7oDcVBV7pVN8R3IZEqmncok8eu5qK/m0l1uxHjZ8xn0/X8vpuM/wAa94qMIG6hgzn/XrTv5d7XIAmVJm6hDYZbjllH/E+h74Mro6K+Xl+hpcRdgja2/mzmekhO41OWHhegkXZGQ81y+kJYd3d+2gJO8LCZy/8YOx3EUzco5CXPKvszzlvRtG0bYllbgkqTeovDCqQZzY+TrHhDZko8lO99zW5wRJ5+Bd1gIUN+GOrhKpT2L8SsRS8LLZQrHF4RNmm9b2mUkvt0lWHhP93f5AmFE2KrB9GPNapwEq53C5+xumj2pv1WVdUZjcwCBWV1u23Gs7pmpyWR/0HuJCMsutpwlAMrnEESztDHY/G5Q/EdJvobyOAcops8LJbtSuKigkzaiBJ00q+QTTV0yrzxNxqXyhEuBxuOBUfYiIcingomPHvRhMJy9E8+tS1K5MftuT/ItUit/DEwJm7AtYuL7munC+U2m6SXqr6Tr50xhJ65TMx4107VBgN1ojafUX
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;6:lsVIf+b6tdCKF/14Sjq5KecvutwkLUDcFgaTK3JdDyjA5sLKcaKmNIv/S4DUnV4qAYmn9n0P60Wq113z1/xmggQR/dMB54hKbhD+YDKw9oXiztqPEChRVNliXFzoWofJyjTwMkkEqZ/QnTtV6V/ptxK1qLJU4SiyDubKtsrrmxZK+NLiX7TpM1MzThol9Eeu8SJCSaUA6+90wMfLdB9BG0uQMsG6xAA/p6WP3VKvOVXR1zeN1RmR0I0BpYEW50ZjZiHLcwmm5a1YD47ghgUbDooIz6LkDHFYm1sgyTeVgE8Z/YibM0AMoJDOw9XUIXVC/gV42v40vFdEIUi3svt0SdOoWNBSvuDR2npXD6WhvSfpTZ+45t9/3fj+oV/nz6hnGJFZF3KOBbNA0InUYDsXz9yU5dQGXvrZOeCfC8QHMfJjljFRcEKeHt0tlo4sUD/EaNS9hCoHse/ARQTqB5vKbNlaYeAY8cn41MiiL10MW8K1kBTewEHxkKO47FY/ucGEJHaZZXVSMbuf1iX5OlbwGACW8B7xrltsmoNEUCsk9hUc5GrccZq3dA2ac7g6UxbLR2PNzQ87KxTIevdmV7z5vZnydlerrJFjfp9agJ63RBCrj38TaTp+XDH4V2nZ8EM1p0UkCuydAsqj574QborT8Cj5DQnzX0Vs/dinBUTFmRVkal8xjHBDFe8ynRAxwnV1jnY8wHD9yNReCsZmzanWUsiNOFeBV0P4mHq6Kn/e7FTx0ORJuAmLVC0FrMHbgKirAiJ4I4CUmIYpyTrlkrxmAzamL+6CsZmxYUkOoNgyBkvIHlJctLtQBNj6tCIOHRw11vYPuLjgVxzsW2ooaX5epSAhIjbzsgRxttEcpInZ222irp5I7Z4si+dNxaw/5LlezJMHV5xT3FdJ0wDB4UrB7Yg31W7UQf4jORJEUMir2rXI1T2Tec+F4BTMVb6OPW7GMjh1TZSteBAfR5O3WTQ4qUCR+NgQJshc5EdLWSIvts0=
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;5:8RlBDohFJDVw250MtlwV7OuSlf+uN3dgSR3pNJIvUlB+glIuysyA2ReYwqkN/LyLwmoCfcrHd6CLDogY8ZgaBYr3ipuwl4DNHmynJHAY/GzqxClYiLYRI6w3VU06MNuFjyN0NqfEIC2Vh28LtNndh+ITNSRahqbNNlKsSoBxfyKu5ePf9U4zv4YONZWBU+QcJNgNqx2augd1K31tNckaCYKF8rN8wtxrxRA9JwOrXnW1mp1lO1jF0woya/8Z2dUrYvlJkk9hmJpt+Gd6wUZYwCdgZljKKICoiwAxnkbJTYdFNoHjaLAUPvB43r1+qllIVVsAHyWm2/YvOVjKq+iTHFsH0jxT7wjrtIDpID7l5/WQHuExvdHuRHV06t5WcuO4vK1nGfy5JKxwe1bTtmWdpwTPkWAhIpKiyo5nsdGHPY7+hcbXaHEwRBUvF24nQwprvq47wXg3LLPFGhxFPWkEKUmakPOviIGb5I783prYzUBcKtrWevwU03PzHzjsXOIC;24:+Hmul05XGyYrVlg6OGnkDNTd4J/gpj7SyzVCa9GCNTo3U6nljiUQx95j7sAP2ewvh58N3w+HcTOaRxfXch6g4Xr6rIYJ1ryMhNXqstJyoIA=
> SpamDiagnosticOutput: 1:99
> SpamDiagnosticMetadata: NSPM
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;7:D9iKnO147gQozMuEAvOtDdaC9pWDMJ2CjdFHTqYR0Zhghn13IZJMgZVzrixkgNIijj8OkTxNHym14PsucSUzp+NXnA37wGLv5QzC5d/1wqYPPmJ/VPPW91kSkWoLXFIaFO1l8pegTzheuxzigPe+ZJFdr0KLfQCLH3+ck7D9kEytS/JbOBONOsva8WMvicsinFhAa2pMNCzb5l13gj3fYRTSse8drseO2yVxvOsjVtSUQXJ70QBiKaxDOXJgU9CDftwHeH3AwNkv9eyVcA8h0HRMXX23v0YKapPvViEt7aKP3krgMbkdP+XTs+QZfEsBDkLDz00qZOGYo1d0AHYHGAIc6zlAxwreKZ18bDefZdhRikWRWZcL0jnw1YPKA1vbyf8wr0gtNNty5eJXVeifBtTx0jc8v9ZC2tjKZHxfAP3oFlU//9ykRk4hYCBiierw/lbb2f79h5sG3ypL3N74FYl2E71qwedzxT6TwTe6c2rCPXoBaGK41lC1JzVddpA8Pp/MgUDCaeN9DWjP3t9a8I+G8rpPLfB6PQLsmnjs1lBXVntiHS1r1ut+Lio2n1ysvsDVAyy6DKVg82ALBKw4TouDqp6dv1BirhGRudedHh4VEelaVkVoQxELWY0IWiRFedo8UtQDKOLAJ5hi529q3xQ6x3BGgR56ApDvP+ArIXT7puwHx6MhGGGjPV5hti8O/QYcSTTmcYXmcZfFtwGi8bh04agwqSNKa9i68MjMPk3LuLKsMRCTAMAmvzZ5EYg+DPFUMeuPsHH8mmuL+IN3zc/pRtGMXjTJDSgWFOKPOa8=
> X-Microsoft-Exchange-Diagnostics:
> 1;CY1PR0101MB0923;20://ocIdbzPfQjObWIZfntUqjWRAEs+PHlZ/yH2QrakFjo/Pph5/KxTUWoe8VXJBfFvlxdhRkWpDcLcS6kJKawmzDLIsFjSpq9qdtJK880zfUEfduKQuUDzn6p1Cjjf60vbwAuluOHjHS9tco2AWBuAr6FgvHvpfL+SGEbAYzrtTM=
> X-OriginatorOrg: cse.ohio-state.edu
> X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jul 2017 21:38:05.7209
> (UTC)
> X-MS-Exchange-CrossTenant-Id: eb095636-1052-4895-952b-1ff9df1d1121
> X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp:
> TenantId=eb095636-1052-4895-952b-1ff9df1d1121;Ip=[128.146.138.10];Helo=[cio-socc-esr05.osuad.osu.edu]
> X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
> X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0101MB0923
> X-Sophos-SenderHistory:
> ip=104.47.40.139,fs=0,da=69427879,mc=1,sc=0,hc=1,sp=0,fso=31567072,re=240,sd=0,hd=0
> X-Sophos-ESA: [sophos1.ldeo.columbia.edu] 4.2.1.0, Antispam-Engine:
> 2.7.2.2107409, Antispam-Data: 2017.7.11.212416
> X-SEA-Spam: Gauge=IIIIIIII, Probability=8%, Report=' RCVD_FROM_IP_DATE
> 0.1, HTML_00_01 0.05, HTML_00_10 0.05, DATE_TZ_NA 0, DKIM_SIGNATURE 0,
> DQ_S_H 0, FROM_EDU_TLD 0, IN_REP_TO 0, LEGITIMATE_SIGNS 0, MSG_THREAD 0,
> NO_URI_HTTPS 0, REFERENCES 0, SPF_NONE 0, WEBMAIL_SOURCE 0, WEBMAIL_XOIP
> 0, WEBMAIL_X_IP_HDR 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0,
> __BOUNCE_NDR_SUBJ_EXEMPT 0, __CC_NAME 0, __CC_NAME_DIFF_FROM_ACC 0,
> __CC_REAL_NAMES 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __DQ_IP_FSO_LARGE
> 0, __DQ_IP_FS_1K 0, __DQ_IP_HIST 0, __DQ_NEG_HEUR 0, __DQ_NEG_IP 0,
> __DQ_S_HIST_1 0, __DQ_S_HIST_2 0, __DQ_S_IP_MC_1 0, __FORWARDED_MSG 0,
> __FRAUD_MONEY_CURRENCY 0, __FRAUD_MONEY_CURRENCY_DOLLAR 0,
> __HAMMY_DAYS_0 0, __HAS_CC_HDR 0, __HAS_FROM 0, __HAS_LIST_HEADER 0,
> __HAS_LIST_HELP 0, __HAS_LIST_ID 0, __HAS_LIST_SUBSCRIBE 0,
> __HAS_LIST_UNSUBSCRIBE 0, __HAS_MSGID 0, __HAS_XOIP 0, __HAS_X_MAILER 0,
> __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_TEXT_P 0, __MIME_TEXT_P1 0,
> __MIME_VERSION 0, __MSGID_APPLEMAIL 0, __NO_HTML_TAG_RAW 0,
> __PHISH_SPEAR_STRUCTURE_1 0, __RDNS_OUTLOOK 0, __REFERENCES 0,
> __SANE_MSGID 0, __SUBJ_ALPHA_END 0, __SUBJ_ALPHA_NEGATE 0,
> __TO_MALFORMED_2 0, __TO_NAME 0, __TO_NAME_DIFF_FROM_ACC 0,
> __TO_REAL_NAMES 0, __URI_NO_WWW 0, __URI_NS , __USER_AGENT_APPLEMAIL 0,
> __X_MAILER_APPLEMAIL 0'
> X-Spam-Score: 0.001 () Limit=8
> DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS
> X-Scanned-By: MIMEDefang 2.79 on 129.236.19.28
> X-Scanned-By: MIMEDefang 2.78
>
> Hi,
>
> Just for clarification, when you say Intranode and Internode, do you
> mean intra-node-inter-vm and inter-node-inter-vm?
>
> Xiaoyi
>
> > On Jul 11, 2017, at 5:25 PM, Maksym Planeta
> <mplaneta at os.inf.tu-dresden.de> wrote:
> >
> >
> >
> > On 07/11/2017 11:14 PM, Xiaoyi Lu wrote:
> >> OK. Let us know the numbers you get if you disable hyper threads.
> >>
> >> For the same NUMA, you can also bound each VM to two cores.
> >
> >
> >> Show us the Inter-VM-Intra-Node numbers for MV2-Virt (def), MV2-Virt
> (explicitly setting IVSHMEM=1), and MV2-Virt (explicitly setting IVSHMEM=0).
> >
> > Here are the results for the default:
> >
> > Latency:
> >
> > | Size | Intranode | Internode |
> > |---------+-----------+-----------|
> > | 0 | 2.55 | 2.92 |
> > | 1 | 2.47 | 2.71 |
> > | 2 | 2.38 | 2.55 |
> > | 4 | 2.33 | 2.45 |
> > | 8 | 2.30 | 2.39 |
> > | 16 | 2.30 | 2.36 |
> > | 32 | 2.35 | 2.39 |
> > | 64 | 2.42 | 2.46 |
> > | 128 | 2.61 | 2.66 |
> > | 256 | 3.75 | 3.78 |
> > | 512 | 4.21 | 4.19 |
> > | 1024 | 5.09 | 5.02 |
> > | 2048 | 5.91 | 5.76 |
> > | 4096 | 8.02 | 7.72 |
> > | 8192 | 11.75 | 11.09 |
> > | 16384 | 21.64 | 20.44 |
> > | 32768 | 34.25 | 31.38 |
> > | 65536 | 59.45 | 52.88 |
> > | 131072 | 109.42 | 96.09 |
> > | 262144 | 209.41 | 182.30 |
> > | 524288 | 409.13 | 354.58 |
> > | 1048576 | 808.61 | 698.58 |
> > | 2097152 | 1619.47 | 1386.03 |
> > | 4194304 | 3428.21 | 2769.11 |
> >
> > Bandwidth:
> >
> > | Size | Intranode | Internode |
> > |---------+-----------+-----------|
> > | 1 | 1.37 | 1.51 |
> > | 2 | 2.71 | 3.02 |
> > | 4 | 5.61 | 6.12 |
> > | 8 | 11.31 | 12.32 |
> > | 16 | 22.66 | 24.93 |
> > | 32 | 44.82 | 49.83 |
> > | 64 | 86.44 | 98.55 |
> > | 128 | 173.66 | 185.80 |
> > | 256 | 255.93 | 319.40 |
> > | 512 | 434.19 | 537.56 |
> > | 1024 | 650.43 | 800.53 |
> > | 2048 | 921.19 | 1078.84 |
> > | 4096 | 1157.34 | 1351.85 |
> > | 8192 | 1219.67 | 1416.09 |
> > | 16384 | 1203.76 | 1398.91 |
> > | 32768 | 1254.92 | 1456.86 |
> > | 65536 | 1282.28 | 1489.19 |
> > | 131072 | 1297.96 | 1507.05 |
> > | 262144 | 1305.38 | 1515.66 |
> > | 524288 | 1308.71 | 1521.66 |
> > | 1048576 | 1310.76 | 1523.69 |
> > | 2097152 | 1311.05 | 1524.68 |
> > | 4194304 | 1307.02 | 1525.13 |
> >
> > The result for the explicit setup I sent in the original email.
> >
> > Just as some additional information, normal ping is a bit faster for
> the local VM:
> >
> > [user at host11 ~]$ ping -c 4 host21
> > PING host21 (141.76.84.9) 56(84) bytes of data.
> > 64 bytes from host21 (141.76.84.9): icmp_seq=1 ttl=64 time=0.487 ms
> > 64 bytes from host21 (141.76.84.9): icmp_seq=2 ttl=64 time=0.480 ms
> > 64 bytes from host21 (141.76.84.9): icmp_seq=3 ttl=64 time=0.498 ms
> > 64 bytes from host21 (141.76.84.9): icmp_seq=4 ttl=64 time=0.478 ms
> >
> > --- host21 ping statistics ---
> > 4 packets transmitted, 4 received, 0% packet loss, time 3000ms
> > rtt min/avg/max/mdev = 0.478/0.485/0.498/0.028 ms
> > [user at host11 ~]$ ping -c 4 host12
> > PING host12 (141.76.84.2) 56(84) bytes of data.
> > 64 bytes from host12 (141.76.84.2): icmp_seq=1 ttl=64 time=0.447 ms
> > 64 bytes from host12 (141.76.84.2): icmp_seq=2 ttl=64 time=0.429 ms
> > 64 bytes from host12 (141.76.84.2): icmp_seq=3 ttl=64 time=0.555 ms
> > 64 bytes from host12 (141.76.84.2): icmp_seq=4 ttl=64 time=0.432 ms
> >
> > --- host12 ping statistics ---
> > 4 packets transmitted, 4 received, 0% packet loss, time 3000ms
> > rtt min/avg/max/mdev = 0.429/0.465/0.555/0.058 ms
> >
> >
> >>
> >> We can then discuss from there.
> >>
> >> Thanks,
> >> Xiaoyi
> >>
> >>> On Jul 11, 2017, at 4:57 PM, Maksym Planeta
> <mplaneta at os.inf.tu-dresden.de> wrote:
> >>>
> >>>
> >>>
> >>> On 07/11/2017 10:44 PM, Xiaoyi Lu wrote:
> >>>> Hi,
> >>>>
> >>>> Thanks for sending us the details.
> >>>>
> >>>> Can you first disable hyper threads?
> >>>>
> >>>
> >>> I can try to repeat the same thing tomorrow without hyperthreads,
> but actually nothing was running there (I was looking with htop).
> >>>
> >>>> And you can try to bound each VM to two cores in the same node.
> >>>
> >>> Do you mean the same NUMA domain? If yes, then I have only one NUMA
> domain. If not then I did this for previous email and saw no difference.
> >>>
> >>>> After that, let us know how the numbers look like for the cases of
> >>>> IVSHMEM enabled and disabled.
> >>>
> >>>
> >>>
> >>>> Thanks,
> >>>> Xiaoyi
> >>>>
> >>>>> On Jul 11, 2017, at 4:20 PM, Maksym Planeta
> <mplaneta at os.inf.tu-dresden.de> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 07/11/2017 10:08 PM, Xiaoyi Lu wrote:
> >>>>>> Hi, Maksym,
> >>>>>>
> >>>>>> Thanks for your feedback.
> >>>>>>
> >>>>>> Can you please let us know your system configurations? Like KVM,
> QEMU versions, cpu info, memory size, HCA, etc. How many VMs and number
> of processes per VM are run on your system?
> >>>>>>
> >>>>>
> >>>>> QEMU/KVM:
> >>>>> $ qemu-system-x86_64 --version
> >>>>> QEMU emulator version 2.8.1(Debian 1:2.8+dfsg-6)
> >>>>>
> >>>>> Kernel:
> >>>>> $ uname -a
> >>>>> Linux ib1 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u1
> (2017-06-18) x86_64 GNU/Linux
> >>>>>
> >>>>> CPU 4 cores + 4 hyperthreads:
> >>>>>
> >>>>> $ cat /proc/cpuinfo
> >>>>> processor : 0
> >>>>> vendor_id : GenuineIntel
> >>>>> cpu family : 6
> >>>>> model : 60
> >>>>> model name : Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
> >>>>> stepping : 3
> >>>>> microcode : 0x9
> >>>>> cpu MHz : 799.987
> >>>>> cache size : 8192 KB
> >>>>> physical id : 0
> >>>>> siblings : 8
> >>>>> core id : 0
> >>>>> cpu cores : 4
> >>>>> apicid : 0
> >>>>> initial apicid : 0
> >>>>> fpu : yes
> >>>>> fpu_exception : yes
> >>>>> cpuid level : 13
> >>>>> wp : yes
> >>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
> nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor
> ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2
> x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm
> abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle
> avx2 smep bmi2 erms invpcid rtm xsaveopt dtherm arat pln pts
> >>>>> bugs :
> >>>>> bogomips : 6799.83
> >>>>> clflush size : 64
> >>>>> cache_alignment : 64
> >>>>> address sizes : 39 bits physical, 48 bits virtual
> >>>>> power management:
> >>>>>
> >>>>> $ free -h
> >>>>> total used free shared buff/cache
> available
> >>>>> Mem: 15G 4.6G 10G 270M
> 814M 10G
> >>>>> Swap: 3.7G 0B 3.7G
> >>>>>
> >>>>> HCA with 4 virtual functions:
> >>>>>
> >>>>> 05:00.0 Network controller: Mellanox Technologies MT27500 Family
> [ConnectX-3]
> >>>>>
> >>>>> # ibv_devinfo
> >>>>> hca_id: mlx4_0
> >>>>> transport: InfiniBand (0)
> >>>>> fw_ver: 2.34.5000
> >>>>> node_guid: f452:1403:0010:a4a0
> >>>>> sys_image_guid: f452:1403:0010:a4a3
> >>>>> vendor_id: 0x02c9
> >>>>> vendor_part_id: 4099
> >>>>> hw_ver: 0x0
> >>>>> board_id: MT_1090120019
> >>>>> phys_port_cnt: 2
> >>>>> port: 1
> >>>>> state: PORT_DOWN (1)
> >>>>> max_mtu: 2048 (4)
> >>>>> active_mtu: 2048 (4)
> >>>>> sm_lid: 0
> >>>>> port_lid: 0
> >>>>> port_lmc: 0x00
> >>>>> link_layer: InfiniBand
> >>>>>
> >>>>> port: 2
> >>>>> state: PORT_ACTIVE (4)
> >>>>> max_mtu: 2048 (4)
> >>>>> active_mtu: 2048 (4)
> >>>>> sm_lid: 3
> >>>>> port_lid: 3
> >>>>> port_lmc: 0x00
> >>>>> link_layer: InfiniBand
> >>>>>
> >>>>>
> >>>>> For this test I had two VMs at each host. For each test only two
> VMs were used, the rest were idling. I didn't do pinning.
> >>>>>
> >>>>> When I pinned the VMs there was no change. Pinning inside VMs is
> disabled, but there is only one vCPU.
> >>>>>
> >>>>> In the attachment virsh dumpxml for one of the VMs.
> >>>>>
> >>>>> Motherboard:
> >>>>>
> >>>>> Manufacturer: Gigabyte Technology Co., Ltd.
> >>>>> Product Name: Z87-HD3
> >>>>>
> >>>>>
> >>>>>> Thanks,
> >>>>>> Xiaoyi
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Regards,
> >>>>> Maksym Planeta
> >>>>> <dump.xml>
> >>>>
> >>>
> >>> --
> >>> Regards,
> >>> Maksym Planeta
> >>>
> >>
> >
> > --
> > Regards,
> > Maksym Planeta
> >
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
More information about the mvapich-discuss
mailing list