path: root/0024-V8-0-4-PCI-vmd-Enable-PCIe-ASPM-and-LTR-on-select-hardware.patch

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id AC0A9C4332F
	for <linux-pci@archiver.kernel.org>; Thu,  3 Nov 2022 02:18:26 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S229954AbiKCCSZ (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 2 Nov 2022 22:18:25 -0400
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56134 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S229436AbiKCCSY (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 2 Nov 2022 22:18:24 -0400
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20F35E81;
        Wed,  2 Nov 2022 19:18:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1667441903; x=1698977903;
  h=from:to:cc:subject:date:message-id:mime-version:
   content-transfer-encoding;
  bh=dRTGogFr6yFQWzAu44s2Shz3Tuu+kZ6YaOpo+8VRwtE=;
  b=DMcLu2BjDmudMN0UTyyA6FzOfZelhaQ8aXzibcjvThwM3TFZjGWBNUKD
   wD87ecQD9wgkVeT9M0zQMQHf/V4yToaol98pmgAsEtm240ZjfYRyDfQly
   +NXTn3/83rPUqYemS0WbI7mo8eLfSBqFbrU+1UlW3xI6xLFAjkgRLVzcD
   lmBHge6U5cuhjmdnCeRUWNDjVAsqPDdObXWhbzy9XAVtP9ErHwqBTLJIf
   36fdHDJUNSglpacZiiJMinGpULwMqK6jKiWcXilzqmkjsqPfVl2BYXgG5
   oPLGYSWzARxBuTWWDlcWEmjHb9o7CLwNLG1Z8LkoRGkou1I6HEtWgummX
   A==;
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="395875409"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="395875409"
Received: from orsmga004.jf.intel.com ([10.7.209.38])
  by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Nov 2022 19:18:22 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="759771646"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="759771646"
Received: from linux.intel.com ([10.54.29.200])
  by orsmga004.jf.intel.com with ESMTP; 02 Nov 2022 19:18:22 -0700
Received: from debox1-desk4.intel.com (unknown [10.212.195.54])
        by linux.intel.com (Postfix) with ESMTP id 38146580D42;
        Wed,  2 Nov 2022 19:18:22 -0700 (PDT)
From:   "David E. Box" <david.e.box@linux.intel.com>
To:     nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev,
        lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com,
        robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com,
        michael.a.bottini@intel.com, rafael@kernel.org, me@adhityamohan.in
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH V8 0/4] PCI: vmd: Enable PCIe ASPM and LTR on select hardware
Date:   Wed,  2 Nov 2022 19:18:18 -0700
Message-Id: <20221103021822.308586-1-david.e.box@linux.intel.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

This series adds a work around for enabling PCIe ASPM and for setting PCIe
LTR values on VMD reserved root ports on select platforms. While
configuration of these capabilities is usually done by BIOS, on these
platforms these capabilities will not be configured because the ports are
not visible to BIOS. This was part of an initial design that expected the
driver to completely handle the ports, including power management. However
on Linux those ports are still managed by the PCIe core, which has the
expectation that they adhere to device standards including BIOS
configuration, leading to this problem.

The target platforms are Tiger Lake, Alder Lake, and Raptor Lake though the
latter has already implemented support for configuring the LTR values.
Meteor Lake is expected add BIOS ASPM support, eliminating the future need
for this work around.

Note, the driver programs the LTRs because BIOS would also normally do this
for devices that do not set them by default. Without this, SoC power
management would be blocked on those platform. This SoC specific value is
the maximum latency required to allow the SoC to enter the deepest power
state.

This patch addresses the following open bugzillas on VMD enabled laptops
that cannot enter low power states.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=212355
Link: https://bugzilla.kernel.org/show_bug.cgi?id=215063
Link: https://bugzilla.kernel.org/show_bug.cgi?id=213717

David E. Box (3):
  PCI: vmd: Use PCI_VDEVICE in device list
  PCI: vmd: Create feature grouping for client products
  PCI: vmd: Add quirk to configure PCIe ASPM and LTR

Michael Bottini (1):
  PCI/ASPM: Add pci_enable_link_state()

 drivers/pci/controller/vmd.c | 96 ++++++++++++++++++++++++++----------
 drivers/pci/pcie/aspm.c      | 54 ++++++++++++++++++++
 include/linux/pci.h          |  7 +++
 3 files changed, 131 insertions(+), 26 deletions(-)


base-commit: 247f34f7b80357943234f93f247a1ae6b6c3a740
-- 
2.25.1


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id E6F34C433FE
	for <linux-pci@archiver.kernel.org>; Thu,  3 Nov 2022 02:18:29 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S230314AbiKCCS2 (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 2 Nov 2022 22:18:28 -0400
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56158 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S230207AbiKCCS1 (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 2 Nov 2022 22:18:27 -0400
Received: from mga07.intel.com (mga07.intel.com [134.134.136.100])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 721852618;
        Wed,  2 Nov 2022 19:18:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1667441905; x=1698977905;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=vzUI8zH+fYLJwczHSdbiw1/5l9LlQMI1ESll22EfjJI=;
  b=QmTz1hTHIb+yN2slhY+HSOi6ZVrWLgup6gAHG1HZEUvZ8Opy1vOjOIY2
   gKpgQkAc1uo1WaKGsdQYFX+Yy/iq1MHzJgiiW2Jex5v4yW0QJd7VqrONf
   8WIQLCNduJnojVdWvt6nau5D9/c48X3KgTE2Tp3/CQ8eiUR6VxqFm9S8g
   RViAPgyzk+++taagSjfQuZ6NJ9/alBW7e4FIvJanRsseUdzyK5MsPFI9P
   WSXMt650HNIjiK9WB8UDB0+bWM3tu7AV7nCkB5ljpWDBfnrbOtITGyNW/
   Zj8+rbcuN44uwYh6KV/09XY9xIq02Q+HNVKjB4OUEKNJWuTZBbEl/43bd
   g==;
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="373794791"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="373794791"
Received: from orsmga008.jf.intel.com ([10.7.209.65])
  by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Nov 2022 19:18:23 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="665784357"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="665784357"
Received: from linux.intel.com ([10.54.29.200])
  by orsmga008.jf.intel.com with ESMTP; 02 Nov 2022 19:18:22 -0700
Received: from debox1-desk4.intel.com (unknown [10.212.195.54])
        by linux.intel.com (Postfix) with ESMTP id 94B3B580B9E;
        Wed,  2 Nov 2022 19:18:22 -0700 (PDT)
From:   "David E. Box" <david.e.box@linux.intel.com>
To:     nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev,
        lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com,
        robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com,
        michael.a.bottini@intel.com, rafael@kernel.org, me@adhityamohan.in
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH V8 1/4] PCI/ASPM: Add pci_enable_link_state()
Date:   Wed,  2 Nov 2022 19:18:19 -0700
Message-Id: <20221103021822.308586-2-david.e.box@linux.intel.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20221103021822.308586-1-david.e.box@linux.intel.com>
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

From: Michael Bottini <michael.a.bottini@linux.intel.com>

Add pci_enable_link_state() to allow devices to change the default BIOS
configured states. Clears the BIOS default settings then sets the new
states and reconfigures the link under the semaphore. Also add
PCIE_LINK_STATE_ALL macro for convenience for callers that want to enable
all link states.

Signed-off-by: Michael Bottini <michael.a.bottini@linux.intel.com>
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
---
 V8
  - No change

 V7
  - Fix description as suggested by Bjorn
  - Rename function to pci_enable_link_state

 V6
  - No change
 V5
  - Rename to pci_enable_default_link_state and model after
    pci_disable_link_state() as suggested by Bjorn.
  - Add helper PCIE_LINK_STATE_ALL which sets bits for all links states and
    clock pm.
  - Clarify commit language to indicate the function changes the default
    link states (not ASPM policy).
 V4
  - Refactor vmd_enable_apsm() to exit early, making the lines shorter
    and more readable. Suggested by Christoph.
 V3
  - No changes
 V2
  - Use return status to print pci_info message if ASPM cannot be enabled.
  - Add missing static declaration, caught by lkp@intel.com

 drivers/pci/pcie/aspm.c | 54 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  7 ++++++
 2 files changed, 61 insertions(+)

diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index 53a1fa306e1e..339c686a5094 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -1181,6 +1181,60 @@ int pci_disable_link_state(struct pci_dev *pdev, int state)
 }
 EXPORT_SYMBOL(pci_disable_link_state);
 
+/**
+ * pci_enable_link_state - Clear and set the default device link state so that
+ * the link may be allowed to enter the specified states. Note that if the
+ * BIOS didn't grant ASPM control to the OS, this does nothing because we can't
+ * touch the LNKCTL register. Also note that this does not enable states
+ * disabled by pci_disable_link_state(). Return 0 or a negative errno.
+ *
+ * @pdev: PCI device
+ * @state: Mask of ASPM link states to enable
+ */
+int pci_enable_link_state(struct pci_dev *pdev, int state)
+{
+	struct pcie_link_state *link = pcie_aspm_get_link(pdev);
+
+	if (!link)
+		return -EINVAL;
+	/*
+	 * A driver requested that ASPM be enabled on this device, but
+	 * if we don't have permission to manage ASPM (e.g., on ACPI
+	 * systems we have to observe the FADT ACPI_FADT_NO_ASPM bit and
+	 * the _OSC method), we can't honor that request.
+	 */
+	if (aspm_disabled) {
+		pci_warn(pdev, "can't override BIOS ASPM; OS doesn't have ASPM control\n");
+		return -EPERM;
+	}
+
+	down_read(&pci_bus_sem);
+	mutex_lock(&aspm_lock);
+	link->aspm_default = 0;
+	if (state & PCIE_LINK_STATE_L0S)
+		link->aspm_default |= ASPM_STATE_L0S;
+	if (state & PCIE_LINK_STATE_L1)
+		/* L1 PM substates require L1 */
+		link->aspm_default |= ASPM_STATE_L1 | ASPM_STATE_L1SS;
+	if (state & PCIE_LINK_STATE_L1_1)
+		link->aspm_default |= ASPM_STATE_L1_1;
+	if (state & PCIE_LINK_STATE_L1_2)
+		link->aspm_default |= ASPM_STATE_L1_2;
+	if (state & PCIE_LINK_STATE_L1_1_PCIPM)
+		link->aspm_default |= ASPM_STATE_L1_1_PCIPM;
+	if (state & PCIE_LINK_STATE_L1_2_PCIPM)
+		link->aspm_default |= ASPM_STATE_L1_2_PCIPM;
+	pcie_config_aspm_link(link, policy_to_aspm_state(link));
+
+	link->clkpm_default = (state & PCIE_LINK_STATE_CLKPM) ? 1 : 0;
+	pcie_set_clkpm(link, policy_to_clkpm_state(link));
+	mutex_unlock(&aspm_lock);
+	up_read(&pci_bus_sem);
+
+	return 0;
+}
+EXPORT_SYMBOL(pci_enable_link_state);
+
 static int pcie_aspm_set_policy(const char *val,
 				const struct kernel_param *kp)
 {
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 2bda4a4e47e8..8c35f15e6012 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1651,10 +1651,15 @@ extern bool pcie_ports_native;
 #define PCIE_LINK_STATE_L1_2		BIT(4)
 #define PCIE_LINK_STATE_L1_1_PCIPM	BIT(5)
 #define PCIE_LINK_STATE_L1_2_PCIPM	BIT(6)
+#define PCIE_LINK_STATE_ALL		(PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1 |\
+					 PCIE_LINK_STATE_CLKPM | PCIE_LINK_STATE_L1_1 |\
+					 PCIE_LINK_STATE_L1_2 | PCIE_LINK_STATE_L1_1_PCIPM |\
+					 PCIE_LINK_STATE_L1_2_PCIPM)
 
 #ifdef CONFIG_PCIEASPM
 int pci_disable_link_state(struct pci_dev *pdev, int state);
 int pci_disable_link_state_locked(struct pci_dev *pdev, int state);
+int pci_enable_link_state(struct pci_dev *pdev, int state);
 void pcie_no_aspm(void);
 bool pcie_aspm_support_enabled(void);
 bool pcie_aspm_enabled(struct pci_dev *pdev);
@@ -1663,6 +1668,8 @@ static inline int pci_disable_link_state(struct pci_dev *pdev, int state)
 { return 0; }
 static inline int pci_disable_link_state_locked(struct pci_dev *pdev, int state)
 { return 0; }
+static inline int pci_enable_link_state(struct pci_dev *pdev, int state)
+{ return 0; }
 static inline void pcie_no_aspm(void) { }
 static inline bool pcie_aspm_support_enabled(void) { return false; }
 static inline bool pcie_aspm_enabled(struct pci_dev *pdev) { return false; }
-- 
2.25.1


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 62CFCC43219
	for <linux-pci@archiver.kernel.org>; Thu,  3 Nov 2022 02:18:31 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S230384AbiKCCS3 (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 2 Nov 2022 22:18:29 -0400
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56162 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S230274AbiKCCS1 (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 2 Nov 2022 22:18:27 -0400
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 561802619;
        Wed,  2 Nov 2022 19:18:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1667441905; x=1698977905;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=lryJqhW0QO99aVIBqjyXzXFrtjj3i89vRFLmrilFP90=;
  b=Df8sKKhA92/hom2M6lfSMoLfzheO17ovcECn6UnhcRkljtkYM5G8Bwti
   4imwXXERTkouM4GDl5zIi7807l2h0WaF1hhK7FnrEfbnnslMLN7fo/GOx
   tJX3o2Rmi4D1MijqjZ8OCNbIyfbjpnnrsXK87bp9Yh8yDXM5SM9qYFCtW
   znugRhwhEZSBAnZJwHjonuaSm+FRxtMHCRCiKW9wVvcnsyJ4QV8npONQR
   sWm5W8mWbM0jX677X3OLQWQjTVdpav3LYiTQ/yoIniJvhM3A6eGA2dI7W
   epYMuCTrGWAZh4qY1C0+5X59xMO5LdSbZSUkFc1uPJsshsWZahmc7sb/f
   w==;
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="395875415"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="395875415"
Received: from orsmga004.jf.intel.com ([10.7.209.38])
  by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Nov 2022 19:18:23 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="759771651"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="759771651"
Received: from linux.intel.com ([10.54.29.200])
  by orsmga004.jf.intel.com with ESMTP; 02 Nov 2022 19:18:23 -0700
Received: from debox1-desk4.intel.com (unknown [10.212.195.54])
        by linux.intel.com (Postfix) with ESMTP id F23D7580DBD;
        Wed,  2 Nov 2022 19:18:22 -0700 (PDT)
From:   "David E. Box" <david.e.box@linux.intel.com>
To:     nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev,
        lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com,
        robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com,
        michael.a.bottini@intel.com, rafael@kernel.org, me@adhityamohan.in
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH V8 2/4] PCI: vmd: Use PCI_VDEVICE in device list
Date:   Wed,  2 Nov 2022 19:18:20 -0700
Message-Id: <20221103021822.308586-3-david.e.box@linux.intel.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20221103021822.308586-1-david.e.box@linux.intel.com>
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

Refactor the PCI ID list to use PCI_VDEVICE.

Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Reviewed-by: Jon Derrick <jonathan.derrick@linux.dev>
Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>
---
 V8 - No change

 V7 - New Patch. Separate patch suggested by Lorenzo

 drivers/pci/controller/vmd.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index e06e9f4fc50f..9dedca714c18 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -994,33 +994,33 @@ static int vmd_resume(struct device *dev)
 static SIMPLE_DEV_PM_OPS(vmd_dev_pm_ops, vmd_suspend, vmd_resume);
 
 static const struct pci_device_id vmd_ids[] = {
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_201D),
+	{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_201D),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP,},
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0),
+	{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW |
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_CAN_BYPASS_MSI_REMAP,},
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x467f),
+	{PCI_VDEVICE(INTEL, 0x467f),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_OFFSET_FIRST_VECTOR,},
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x4c3d),
+	{PCI_VDEVICE(INTEL, 0x4c3d),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_OFFSET_FIRST_VECTOR,},
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xa77f),
+	{PCI_VDEVICE(INTEL, 0xa77f),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_OFFSET_FIRST_VECTOR,},
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x7d0b),
+	{PCI_VDEVICE(INTEL, 0x7d0b),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_OFFSET_FIRST_VECTOR,},
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xad0b),
+	{PCI_VDEVICE(INTEL, 0xad0b),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_OFFSET_FIRST_VECTOR,},
-	{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
+	{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
 		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_OFFSET_FIRST_VECTOR,},
-- 
2.25.1


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 2C015C4332F
	for <linux-pci@archiver.kernel.org>; Thu,  3 Nov 2022 02:18:35 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S230353AbiKCCSa (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 2 Nov 2022 22:18:30 -0400
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56160 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S230266AbiKCCS1 (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 2 Nov 2022 22:18:27 -0400
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 555FCE81;
        Wed,  2 Nov 2022 19:18:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1667441905; x=1698977905;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=HPzcsAi/d9ZZUHE0bfR05SfFvSoDaIB+WYmjKiFqpy4=;
  b=V33lj/7ygFrkVPShV+DmuCKKknUkX0Q7z9HbHDxn3ShOrOfFS2Rx4nM3
   xjl0IwcP/rEPGdhppTqNFW5NIerdyqTnmdg7YyQGxPyRTucNImawYYZRW
   w/yiahAjImxpdSl7IS2lptgcDAGf47ICAq5JErYk1rvNLF39QLspv62DR
   CC1HN6W1abC4BX/aRlEjSpdFE1vBYY6p1K5XihseccLQ3scRbxNXZHfce
   ultbE/3pOXffkDVu0W9FauUZg7J2fpEAkxtc27cyKbQ9k99h8Bfhg/3cM
   +CYz+uMa0lzXX/m6KXst/qVnsCsoPJb2lYrXIcHu+Q46hgUJKrph3wKRX
   g==;
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="292886416"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="292886416"
Received: from orsmga007.jf.intel.com ([10.7.209.58])
  by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Nov 2022 19:18:23 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="629175111"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="629175111"
Received: from linux.intel.com ([10.54.29.200])
  by orsmga007.jf.intel.com with ESMTP; 02 Nov 2022 19:18:23 -0700
Received: from debox1-desk4.intel.com (unknown [10.212.195.54])
        by linux.intel.com (Postfix) with ESMTP id 56E3E580B9E;
        Wed,  2 Nov 2022 19:18:23 -0700 (PDT)
From:   "David E. Box" <david.e.box@linux.intel.com>
To:     nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev,
        lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com,
        robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com,
        michael.a.bottini@intel.com, rafael@kernel.org, me@adhityamohan.in
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH V8 3/4] PCI: vmd: Create feature grouping for client products
Date:   Wed,  2 Nov 2022 19:18:21 -0700
Message-Id: <20221103021822.308586-4-david.e.box@linux.intel.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20221103021822.308586-1-david.e.box@linux.intel.com>
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

Simplify the device ID list by creating a grouping of features shared by
client products.

Suggested-by: Jon Derrick <jonathan.derrick@linux.dev>
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
---

 V8 - New patch.

 drivers/pci/controller/vmd.c | 28 ++++++++++------------------
 1 file changed, 10 insertions(+), 18 deletions(-)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 9dedca714c18..86f3085db014 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -68,6 +68,10 @@ enum vmd_features {
 	VMD_FEAT_CAN_BYPASS_MSI_REMAP		= (1 << 4),
 };
 
+#define VMD_FEATS_CLIENT	(VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |	\
+				 VMD_FEAT_HAS_BUS_RESTRICTIONS |	\
+				 VMD_FEAT_OFFSET_FIRST_VECTOR)
+
 static DEFINE_IDA(vmd_instance_ida);
 
 /*
@@ -1001,29 +1005,17 @@ static const struct pci_device_id vmd_ids[] = {
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_CAN_BYPASS_MSI_REMAP,},
 	{PCI_VDEVICE(INTEL, 0x467f),
-		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
-				VMD_FEAT_HAS_BUS_RESTRICTIONS |
-				VMD_FEAT_OFFSET_FIRST_VECTOR,},
+		.driver_data = VMD_FEATS_CLIENT,},
 	{PCI_VDEVICE(INTEL, 0x4c3d),
-		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
-				VMD_FEAT_HAS_BUS_RESTRICTIONS |
-				VMD_FEAT_OFFSET_FIRST_VECTOR,},
+		.driver_data = VMD_FEATS_CLIENT,},
 	{PCI_VDEVICE(INTEL, 0xa77f),
-		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
-				VMD_FEAT_HAS_BUS_RESTRICTIONS |
-				VMD_FEAT_OFFSET_FIRST_VECTOR,},
+		.driver_data = VMD_FEATS_CLIENT,},
 	{PCI_VDEVICE(INTEL, 0x7d0b),
-		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
-				VMD_FEAT_HAS_BUS_RESTRICTIONS |
-				VMD_FEAT_OFFSET_FIRST_VECTOR,},
+		.driver_data = VMD_FEATS_CLIENT,},
 	{PCI_VDEVICE(INTEL, 0xad0b),
-		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
-				VMD_FEAT_HAS_BUS_RESTRICTIONS |
-				VMD_FEAT_OFFSET_FIRST_VECTOR,},
+		.driver_data = VMD_FEATS_CLIENT,},
 	{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
-		.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
-				VMD_FEAT_HAS_BUS_RESTRICTIONS |
-				VMD_FEAT_OFFSET_FIRST_VECTOR,},
+		.driver_data = VMD_FEATS_CLIENT,},
 	{0,}
 };
 MODULE_DEVICE_TABLE(pci, vmd_ids);
-- 
2.25.1


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 35523C43217
	for <linux-pci@archiver.kernel.org>; Thu,  3 Nov 2022 02:18:36 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S230466AbiKCCSd (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 2 Nov 2022 22:18:33 -0400
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56188 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S229436AbiKCCS2 (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 2 Nov 2022 22:18:28 -0400
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 119BE13D48;
        Wed,  2 Nov 2022 19:18:27 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1667441907; x=1698977907;
  h=from:to:cc:subject:date:message-id:in-reply-to:
   references:mime-version:content-transfer-encoding;
  bh=aEiHMG/t7yyno42B1YTG/JN48w91eX/H9maLsZ24Sp0=;
  b=mMsfE7DPPMYcfi/Ii0kcUlolsfkUNNfdpDmA7EcRKiHEqsMbCdeVtIRk
   Qs5CyXpnBkA+g4c0G+x9fNEu1vWUsI+KTzOrr8z1OZz/DMq0WfHO+gKcz
   mdEZ75Y2AE4aUxEYeyqsGTU6FpnZ+WdR3jBbPoVBp1AcyA58dVZwMkSjs
   UB3PQfdysXnrfSdgMTzZMlsBsSmohnLAtn1BeKgdZtH2AHPxO6wsv9TPY
   NhcRZKWd2tVPfq77Qol0XDRGQZUdWXPFZfJHQ2b2GXkeCJ75rrHeEHVvU
   Rfh+HEb7GDuHamdtXYPnlkaA183sJycUDR+Vv8rYICq0j7o7a6krxEExc
   A==;
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="292886417"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="292886417"
Received: from orsmga003.jf.intel.com ([10.7.209.27])
  by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Nov 2022 19:18:24 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6500,9779,10519"; a="585619408"
X-IronPort-AV: E=Sophos;i="5.95,235,1661842800"; 
   d="scan'208";a="585619408"
Received: from linux.intel.com ([10.54.29.200])
  by orsmga003.jf.intel.com with ESMTP; 02 Nov 2022 19:18:24 -0700
Received: from debox1-desk4.intel.com (unknown [10.212.195.54])
        by linux.intel.com (Postfix) with ESMTP id B46F4580DBD;
        Wed,  2 Nov 2022 19:18:23 -0700 (PDT)
From:   "David E. Box" <david.e.box@linux.intel.com>
To:     nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev,
        lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com,
        robh@kernel.org, bhelgaas@google.com, david.e.box@linux.intel.com,
        michael.a.bottini@intel.com, rafael@kernel.org, me@adhityamohan.in
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH V8 4/4] PCI: vmd: Add quirk to configure PCIe ASPM and LTR
Date:   Wed,  2 Nov 2022 19:18:22 -0700
Message-Id: <20221103021822.308586-5-david.e.box@linux.intel.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20221103021822.308586-1-david.e.box@linux.intel.com>
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

PCIe ports reserved for VMD use are not visible to BIOS and therefore not
configured to enable PCIe ASPM or LTR values (which BIOS will configure if
they are not set). Lack of this programming results in high power
consumption on laptops as reported in bugzilla.  For affected products use
pci_enable_link_state to set the allowed link states for devices on the
root ports. Also set the LTR value to the maximum value needed for the SoC.

This is a workaround for products from Rocket Lake through Alder Lake.
Raptor Lake, the latest product at this time, has already implemented LTR
configuring in BIOS. Future products will move ASPM configuration back to
BIOS as well.  As this solution is intended for laptops, support is not
added for hotplug or for devices downstream of a switch on the root port.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=212355
Link: https://bugzilla.kernel.org/show_bug.cgi?id=215063
Link: https://bugzilla.kernel.org/show_bug.cgi?id=213717

Signed-off-by: Michael Bottini <michael.a.bottini@linux.intel.com>
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Reviewed-by: Jon Derrick <jonathan.derrick@linux.dev>
Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>
---
 V8
  - Removed struct vmd_device_data patch. Instead use #define for the LTR
    value which is the same across all products needing the quirk.
 V7
  - No change
 V6
  - Set ASPM first before setting LTR. This is needed because some
    devices may only have LTR set by BIOS and not ASPM
  - Skip setting the LTR if the current LTR in non-zero.
 V5
  - Provide the LTR value as driver data.
  - Use DWORD for the config space write to avoid PCI WORD access bug.
  - Set ASPM links firsts, enabling all link states, before setting a
    default LTR if the capability is present
  - Add kernel message that VMD is setting the device LTR.
 V4
  - Refactor vmd_enable_apsm() to exit early, making the lines shorter
    and more readable. Suggested by Christoph.
 V3
  - No changes
 V2
  - Use return status to print pci_info message if ASPM cannot be enabled.
  - Add missing static declaration, caught by lkp@intel.com
 drivers/pci/controller/vmd.c | 64 ++++++++++++++++++++++++++++++++----
 1 file changed, 58 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 86f3085db014..cba57e3091f6 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -66,12 +66,22 @@ enum vmd_features {
 	 * interrupt handling.
 	 */
 	VMD_FEAT_CAN_BYPASS_MSI_REMAP		= (1 << 4),
+
+	/*
+	 * Enable ASPM on the PCIE root ports and set the default LTR of the
+	 * storage devices on platforms where these values are not configured by
+	 * BIOS. This is needed for laptops, which require these settings for
+	 * proper power management of the SoC.
+	 */
+	VMD_FEAT_BIOS_PM_QUIRK		= (1 << 5),
 };
 
 #define VMD_FEATS_CLIENT	(VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |	\
 				 VMD_FEAT_HAS_BUS_RESTRICTIONS |	\
 				 VMD_FEAT_OFFSET_FIRST_VECTOR)
 
+#define VMD_BIOS_PM_QUIRK_LTR	0x1003	/* 3145728 ns */
+
 static DEFINE_IDA(vmd_instance_ida);
 
 /*
@@ -713,6 +723,46 @@ static void vmd_copy_host_bridge_flags(struct pci_host_bridge *root_bridge,
 	vmd_bridge->native_dpc = root_bridge->native_dpc;
 }
 
+/*
+ * Enable ASPM and LTR settings on devices that aren't configured by BIOS.
+ */
+static int vmd_pm_enable_quirk(struct pci_dev *pdev, void *userdata)
+{
+	unsigned long features = *(unsigned long *)userdata;
+	u16 ltr = VMD_BIOS_PM_QUIRK_LTR;
+	u32 ltr_reg;
+	int pos;
+
+	if (!(features & VMD_FEAT_BIOS_PM_QUIRK))
+		return 0;
+
+	pci_enable_link_state(pdev, PCIE_LINK_STATE_ALL);
+
+	pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_LTR);
+	if (!pos)
+		return 0;
+
+	/*
+	 * Skip if the max snoop LTR is non-zero, indicating BIOS has set it
+	 * so the LTR quirk is not needed.
+	 */
+	pci_read_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, &ltr_reg);
+	if (!!(ltr_reg & (PCI_LTR_VALUE_MASK | PCI_LTR_SCALE_MASK)))
+		return 0;
+
+	/*
+	 * Set the default values to the maximum required by the platform to
+	 * allow the deepest power management savings. Write as a DWORD where
+	 * the lower word is the max snoop latency and the upper word is the
+	 * max non-snoop latency.
+	 */
+	ltr_reg = (ltr << 16) | ltr;
+	pci_write_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, ltr_reg);
+	pci_info(pdev, "VMD: Default LTR value set by driver\n");
+
+	return 0;
+}
+
 static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 {
 	struct pci_sysdata *sd = &vmd->sysdata;
@@ -867,6 +917,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 		pci_reset_bus(child->self);
 	pci_assign_unassigned_bus_resources(vmd->bus);
 
+	pci_walk_bus(vmd->bus, vmd_pm_enable_quirk, &features);
+
 	/*
 	 * VMD root buses are virtual and don't return true on pci_is_pcie()
 	 * and will fail pcie_bus_configure_settings() early. It can instead be
@@ -1005,17 +1057,17 @@ static const struct pci_device_id vmd_ids[] = {
 				VMD_FEAT_HAS_BUS_RESTRICTIONS |
 				VMD_FEAT_CAN_BYPASS_MSI_REMAP,},
 	{PCI_VDEVICE(INTEL, 0x467f),
-		.driver_data = VMD_FEATS_CLIENT,},
+		.driver_data = VMD_FEATS_CLIENT | VMD_FEAT_BIOS_PM_QUIRK,},
 	{PCI_VDEVICE(INTEL, 0x4c3d),
-		.driver_data = VMD_FEATS_CLIENT,},
+		.driver_data = VMD_FEATS_CLIENT | VMD_FEAT_BIOS_PM_QUIRK,},
 	{PCI_VDEVICE(INTEL, 0xa77f),
-		.driver_data = VMD_FEATS_CLIENT,},
+		.driver_data = VMD_FEATS_CLIENT | VMD_FEAT_BIOS_PM_QUIRK,},
 	{PCI_VDEVICE(INTEL, 0x7d0b),
-		.driver_data = VMD_FEATS_CLIENT,},
+		.driver_data = VMD_FEATS_CLIENT | VMD_FEAT_BIOS_PM_QUIRK,},
 	{PCI_VDEVICE(INTEL, 0xad0b),
-		.driver_data = VMD_FEATS_CLIENT,},
+		.driver_data = VMD_FEATS_CLIENT | VMD_FEAT_BIOS_PM_QUIRK,},
 	{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
-		.driver_data = VMD_FEATS_CLIENT,},
+		.driver_data = VMD_FEATS_CLIENT | VMD_FEAT_BIOS_PM_QUIRK,},
 	{0,}
 };
 MODULE_DEVICE_TABLE(pci, vmd_ids);
-- 
2.25.1


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 9BE9DC43219
	for <linux-pci@archiver.kernel.org>; Wed, 23 Nov 2022 10:43:16 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S236342AbiKWKnN (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 23 Nov 2022 05:43:13 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44260 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S237953AbiKWKmY (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 23 Nov 2022 05:42:24 -0500
Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0F3D769FC;
        Wed, 23 Nov 2022 02:27:25 -0800 (PST)
Received: by mail-pj1-f53.google.com with SMTP id o5-20020a17090a678500b00218cd5a21c9so1425739pjj.4;
        Wed, 23 Nov 2022 02:27:25 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pwpkG9G3xkjOyzikzQ4T/gV3SMpUqELfzhIlK43yLpY=;
        b=jQXLGvDVVqf0LTqxfMq535ZmBr0TkT6RB8p0K8w8T4X78ujUSLrRE1BiKXSaqWNkmw
         cAkayP69927XFP5BW+z1irjuWY5mWxL4XQ507eTgf2AzwJhLDBEP2gNyowTAh+MWX5YJ
         DvlEpF45NCRkz+8Gm7+Cz4tyS2l9/XAiVpA7UMj0ODEclCcxmeXJwkN1+Z/oqKyAeSbw
         3+W9WGzmfnnF16NWn5dQpa3a8WYifXYRCN4bydwZ326GLbvuH8Zdo11ZhikVrjDkZ9vA
         UHat2k1hFjfe0oXg7Vvg1dokDaesqRyGFwEHZSuiPBVZN3X5ZtEvPC9ueBWj3baE/vNn
         vDVg==
X-Gm-Message-State: ANoB5pmNpoOo2LNpR3OgzqqEeSYg/jELrMBlhcX2HhIdcA9xj7wx9aA2
        v2LitFYUz66ji4yIFNDP7sk=
X-Google-Smtp-Source: AA0mqf79gqmDL9stZaRGjrZHLamllSmaeTB+6NNa9TcOtJrVW/NC2LzyXTItQ7A2l8c6qcAq/OLfbA==
X-Received: by 2002:a17:90b:484:b0:218:9d3d:71f4 with SMTP id bh4-20020a17090b048400b002189d3d71f4mr19624100pjb.148.1669199244920;
        Wed, 23 Nov 2022 02:27:24 -0800 (PST)
Received: from [192.168.0.247] (61-227-110-150.dynamic-ip.hinet.net. [61.227.110.150])
        by smtp.gmail.com with ESMTPSA id i8-20020a170902c94800b00172fad607b3sm13969212pla.207.2022.11.23.02.27.20
        (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
        Wed, 23 Nov 2022 02:27:24 -0800 (PST)
Message-ID: <5c766840-e092-45ea-0664-7bbdb78b933a@canonical.com>
Date:   Wed, 23 Nov 2022 18:27:18 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.5.0
Subject: Re: [PATCH V8 0/4] PCI: vmd: Enable PCIe ASPM and LTR on select
 hardware
Content-Language: en-US
To:     "David E. Box" <david.e.box@linux.intel.com>
Cc:     michael.a.bottini@intel.com, linux-pci@vger.kernel.org,
        linux-kernel@vger.kernel.org, me@adhityamohan.in,
        rafael@kernel.org, hch@infradead.org, robh@kernel.org,
        bhelgaas@google.com, kw@linux.com, lorenzo.pieralisi@arm.com,
        nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
From:   You-Sheng Yang <vicamo.yang@canonical.com>
In-Reply-To: <20221103021822.308586-1-david.e.box@linux.intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

Hi David,

On 11/3/22 10:18, David E. Box wrote:
> This series adds a work around for enabling PCIe ASPM and for setting PCIe
> LTR values on VMD reserved root ports on select platforms. While
> configuration of these capabilities is usually done by BIOS, on these
> platforms these capabilities will not be configured because the ports are
> not visible to BIOS. This was part of an initial design that expected the
> driver to completely handle the ports, including power management. However
> on Linux those ports are still managed by the PCIe core, which has the
> expectation that they adhere to device standards including BIOS
> configuration, leading to this problem.
>
> The target platforms are Tiger Lake, Alder Lake, and Raptor Lake though the
> latter has already implemented support for configuring the LTR values.
> Meteor Lake is expected add BIOS ASPM support, eliminating the future need
> for this work around.


It appears to me that this patch series works only on Tiger Lake. We 
have tried to revert our current work-arounds in Ubuntu kernels 
generic-5.15/oem-5.17/oem-6.0/unstable-6.1 and apply this series, the 
prebuilt kernels can be found in:

   https://launchpad.net/~vicamo/+archive/ubuntu/ppa-1996620

However, only TGL can still enter PC10 as before.


ADL-M, RPL platforms will stay in PC3 with vmd LTR set, but ASPM 
disabled. i915 RC6 blocked, too:

$ sudo cat /sys/kernel/debug/dri/

0/i915_dmc_info
...
DC3CO count: 0
DC3 -> DC5 count: 100
DC5 -> DC6 count: 0


> Note, the driver programs the LTRs because BIOS would also normally do this
> for devices that do not set them by default. Without this, SoC power
> management would be blocked on those platform. This SoC specific value is
> the maximum latency required to allow the SoC to enter the deepest power
> state.
>
> This patch addresses the following open bugzillas on VMD enabled laptops
> that cannot enter low power states.
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=212355
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=215063
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=213717
>
> David E. Box (3):
>    PCI: vmd: Use PCI_VDEVICE in device list
>    PCI: vmd: Create feature grouping for client products
>    PCI: vmd: Add quirk to configure PCIe ASPM and LTR
>
> Michael Bottini (1):
>    PCI/ASPM: Add pci_enable_link_state()
>
>   drivers/pci/controller/vmd.c | 96 ++++++++++++++++++++++++++----------
>   drivers/pci/pcie/aspm.c      | 54 ++++++++++++++++++++
>   include/linux/pci.h          |  7 +++
>   3 files changed, 131 insertions(+), 26 deletions(-)
>
>
> base-commit: 247f34f7b80357943234f93f247a1ae6b6c3a740


Regards,
You-Sheng Yang


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 6B305C4332F
	for <linux-pci@archiver.kernel.org>; Wed, 23 Nov 2022 16:10:05 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S238934AbiKWQKD (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 23 Nov 2022 11:10:03 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37158 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S238863AbiKWQKA (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 23 Nov 2022 11:10:00 -0500
Received: from mga11.intel.com (mga11.intel.com [192.55.52.93])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75D25C76BB;
        Wed, 23 Nov 2022 08:09:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1669219796; x=1700755796;
  h=message-id:subject:from:reply-to:to:cc:date:in-reply-to:
   references:content-transfer-encoding:mime-version;
  bh=41Amlkh3nNXXy8dAcUXdlKhcq7idpet7xOPClTf5RII=;
  b=ioLNdzHN/4vRJQKS4hup4Bg24QU8kYQktqJU0/6SW3Y036oCQFdX/bS2
   OV6UHWmdGkS5jwdz+8Avou8tpNLdcs6uqaMFo2eM58+meR9kDDy5pBx6k
   5VxsFkNA+Qpjl70pJaMXqp/bK3oehRmeS5Y6JOUCr20yqF6UP1HDv6quc
   ZSviqMMlMyDW7SIZU2JmboOjAZBUIWR6z8vwVacpSs0+vMDUPWZSo5+NM
   rB5mc00bAq2vtFk2JINllWS/ant/9lPeAoAEbZDXyMStqDj67O453j0es
   jKhxMgnwYtDUD599Kz36cWZoCOarzMTeNUAVcwpv1q2G6NZH3bbB8awqy
   Q==;
X-IronPort-AV: E=McAfee;i="6500,9779,10540"; a="311729547"
X-IronPort-AV: E=Sophos;i="5.96,187,1665471600"; 
   d="scan'208";a="311729547"
Received: from fmsmga001.fm.intel.com ([10.253.24.23])
  by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Nov 2022 08:09:49 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=McAfee;i="6500,9779,10540"; a="784293582"
X-IronPort-AV: E=Sophos;i="5.96,187,1665471600"; 
   d="scan'208";a="784293582"
Received: from linux.intel.com ([10.54.29.200])
  by fmsmga001.fm.intel.com with ESMTP; 23 Nov 2022 08:09:48 -0800
Received: from brpivitt-mobl1.amr.corp.intel.com (unknown [10.252.128.124])
        by linux.intel.com (Postfix) with ESMTP id 61E4E580AFF;
        Wed, 23 Nov 2022 08:09:48 -0800 (PST)
Message-ID: <17b05a72d2db1074cef9d5e9f85b347850f171d5.camel@linux.intel.com>
Subject: Re: [PATCH V8 0/4] PCI: vmd: Enable PCIe ASPM and LTR on select
 hardware
From:   "David E. Box" <david.e.box@linux.intel.com>
Reply-To: david.e.box@linux.intel.com
To:     You-Sheng Yang <vicamo.yang@canonical.com>
Cc:     michael.a.bottini@intel.com, linux-pci@vger.kernel.org,
        linux-kernel@vger.kernel.org, me@adhityamohan.in,
        rafael@kernel.org, hch@infradead.org, robh@kernel.org,
        bhelgaas@google.com, kw@linux.com, lorenzo.pieralisi@arm.com,
        nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev
Date:   Wed, 23 Nov 2022 08:09:48 -0800
In-Reply-To: <5c766840-e092-45ea-0664-7bbdb78b933a@canonical.com>
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
         <5c766840-e092-45ea-0664-7bbdb78b933a@canonical.com>
Organization: David E. Box
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu1 
MIME-Version: 1.0
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

Hi You-Sheng,

On Wed, 2022-11-23 at 18:27 +0800, You-Sheng Yang wrote:
> Hi David,
>=20
> On 11/3/22 10:18, David E. Box wrote:
> > This series adds a work around for enabling PCIe ASPM and for setting P=
CIe
> > LTR values on VMD reserved root ports on select platforms. While
> > configuration of these capabilities is usually done by BIOS, on these
> > platforms these capabilities will not be configured because the ports a=
re
> > not visible to BIOS. This was part of an initial design that expected t=
he
> > driver to completely handle the ports, including power management. Howe=
ver
> > on Linux those ports are still managed by the PCIe core, which has the
> > expectation that they adhere to device standards including BIOS
> > configuration, leading to this problem.
> >=20
> > The target platforms are Tiger Lake, Alder Lake, and Raptor Lake though=
 the
> > latter has already implemented support for configuring the LTR values.
> > Meteor Lake is expected add BIOS ASPM support, eliminating the future n=
eed
> > for this work around.
>=20
>=20
> It appears to me that this patch series works only on Tiger Lake. We=20
> have tried to revert our current work-arounds in Ubuntu kernels=20
> generic-5.15/oem-5.17/oem-6.0/unstable-6.1 and apply this series, the=20
> prebuilt kernels can be found in:
>=20
> =C2=A0=C2=A0 https://launchpad.net/~vicamo/+archive/ubuntu/ppa-1996620
>=20
> However, only TGL can still enter PC10 as before.
>=20
>=20
> ADL-M, RPL platforms will stay in PC3 with vmd LTR set, but ASPM=20
> disabled.

For the patch to work BIOS must allow the OS to control ASPM. If this is no=
t the
case then you will see the message "ACPI FADT declares the system doesn't
support PCIe ASPM, so disable it". Please check for this on the systems tha=
t
don't work. If so the only option is a BIOS change to enable it.

David

>  i915 RC6 blocked, too:
>=20
> $ sudo cat /sys/kernel/debug/dri/
>=20
> 0/i915_dmc_info
> ...
> DC3CO count: 0
> DC3 -> DC5 count: 100
> DC5 -> DC6 count: 0
>=20
>=20
> > Note, the driver programs the LTRs because BIOS would also normally do =
this
> > for devices that do not set them by default. Without this, SoC power
> > management would be blocked on those platform. This SoC specific value =
is
> > the maximum latency required to allow the SoC to enter the deepest powe=
r
> > state.
> >=20
> > This patch addresses the following open bugzillas on VMD enabled laptop=
s
> > that cannot enter low power states.
> >=20
> > Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D212355
> > Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D215063
> > Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D213717
> >=20
> > David E. Box (3):
> > =C2=A0=C2=A0 PCI: vmd: Use PCI_VDEVICE in device list
> > =C2=A0=C2=A0 PCI: vmd: Create feature grouping for client products
> > =C2=A0=C2=A0 PCI: vmd: Add quirk to configure PCIe ASPM and LTR
> >=20
> > Michael Bottini (1):
> > =C2=A0=C2=A0 PCI/ASPM: Add pci_enable_link_state()
> >=20
> > =C2=A0 drivers/pci/controller/vmd.c | 96 ++++++++++++++++++++++++++----=
------
> > =C2=A0 drivers/pci/pcie/aspm.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 54 +++++=
+++++++++++++++
> > =C2=A0 include/linux/pci.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 |=C2=A0 7 +++
> > =C2=A0 3 files changed, 131 insertions(+), 26 deletions(-)
> >=20
> >=20
> > base-commit: 247f34f7b80357943234f93f247a1ae6b6c3a740
>=20
>=20
> Regards,
> You-Sheng Yang
>=20


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id ABA40C433FE
	for <linux-pci@archiver.kernel.org>; Thu, 24 Nov 2022 16:40:17 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S229477AbiKXQkQ (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Thu, 24 Nov 2022 11:40:16 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38952 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S229379AbiKXQkO (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Thu, 24 Nov 2022 11:40:14 -0500
Received: from smtp-relay-internal-1.canonical.com (smtp-relay-internal-1.canonical.com [185.125.188.123])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A39410CE82
        for <linux-pci@vger.kernel.org>; Thu, 24 Nov 2022 08:40:13 -0800 (PST)
Received: from mail-il1-f200.google.com (mail-il1-f200.google.com [209.85.166.200])
        (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
         key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
        (No client certificate requested)
        by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 192763F46A
        for <linux-pci@vger.kernel.org>; Thu, 24 Nov 2022 16:40:11 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com;
        s=20210705; t=1669308011;
        bh=rFBx8o9ACr2aE3UXlk16oilIXDTZ3TK8L+Ksy/y90RU=;
        h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject:
         To:Cc:Content-Type;
        b=b2hIghAdfhN+u1QlWRhqvH49RiP3lpueoPPYUViLjmuWE0Al+czHFB3IQnWqgH1e4
         3Kuj+RqM3hAxbjlThDt2atBTSGcqINNsRKiAaRXT7Et9kUbUABoTWyHX9tgxqqyJ4h
         zLF/C6XbBalqzA3L42ogy5LI20gK/qjDliYBeLrWUTGSz8NRGZQaejSlnpEX2r0YJf
         3uR9Rl+Erbvjs9Gh9NvVEhDHdTtQ8Cp5HQzMgGK7j80ySnkOTt+ww+3fMCJbrpgjIM
         abtRqx8Q2fglJuPIeKb0oG20qd0fizYjUtSSJu17I81ShF+yWduw1O4a/R+5x1dWQW
         tSVXNiAB8JUmQ==
Received: by mail-il1-f200.google.com with SMTP id k3-20020a92c243000000b0030201475a6bso1433616ilo.9
        for <linux-pci@vger.kernel.org>; Thu, 24 Nov 2022 08:40:11 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=rFBx8o9ACr2aE3UXlk16oilIXDTZ3TK8L+Ksy/y90RU=;
        b=5oO1UTG5GWs75QvhfQkgfvfcUjBX8FroWSSYuf6XiLkeBDzXmnkddKhGbLtAZgEKU2
         WDoBxaC831l2zBywqZV3hSEHn9rWwzYYplHdfZ4ruPBzLpE35N1ypD0Y2rarZWHmW/9r
         g/okR9U9zTo/U6pyO6ZRjnLIhe8jXO5axTNcXkdzjC9806gGXh8h9EJd0hjlJaBOqktQ
         7ZZVlm6lpQNmlwnvCm7n4mRwDiE9HOj9UdWJdunbxDOantGvVz23KY4pkEV7aLBrbnlJ
         F+NwHlGvubESgbJuNLnuYtcSmLOld7xNO8SuyVFQHLEFQ9faNHdUaIlF8+IqqImhHk/B
         gHfw==
X-Gm-Message-State: ANoB5pkWthOW13SXoGsLHkDt4M0NKp6SAgiC2OROjsVUmsuLZmxU/K6R
        QCRvV6XJmvtST+Jr6diNvUhD40VH216VUvStymK9NpQA2A1hlf+Qrccxaqy71zDldaGgK/GrTMz
        o+LViVPjFvV036Dt83rks/0i2Gn21kVsURFitSWGSySt9ZgaNynGHVQ==
X-Received: by 2002:a6b:5904:0:b0:6af:ed54:1c81 with SMTP id n4-20020a6b5904000000b006afed541c81mr8426730iob.106.1669308009796;
        Thu, 24 Nov 2022 08:40:09 -0800 (PST)
X-Google-Smtp-Source: AA0mqf6aovE2/ZA6S/DsuJGYDElLduwKMbC8q/hMU3FEu6WeD0yAmkdGpz22Y9xg5ahZfk76UnYd5HasZD8Lb1aFD9U=
X-Received: by 2002:a6b:5904:0:b0:6af:ed54:1c81 with SMTP id
 n4-20020a6b5904000000b006afed541c81mr8426709iob.106.1669308009474; Thu, 24
 Nov 2022 08:40:09 -0800 (PST)
MIME-Version: 1.0
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
 <5c766840-e092-45ea-0664-7bbdb78b933a@canonical.com> <17b05a72d2db1074cef9d5e9f85b347850f171d5.camel@linux.intel.com>
In-Reply-To: <17b05a72d2db1074cef9d5e9f85b347850f171d5.camel@linux.intel.com>
From:   You-Sheng Yang <vicamo.yang@canonical.com>
Date:   Fri, 25 Nov 2022 00:39:58 +0800
Message-ID: <CA+rHWA+7KudasAMnO=dCUSQWb76Eu=DALA=1eLd_jjLw0dEMkA@mail.gmail.com>
Subject: Re: [PATCH V8 0/4] PCI: vmd: Enable PCIe ASPM and LTR on select hardware
To:     david.e.box@linux.intel.com
Cc:     michael.a.bottini@intel.com, linux-pci@vger.kernel.org,
        linux-kernel@vger.kernel.org, me@adhityamohan.in,
        rafael@kernel.org, hch@infradead.org, robh@kernel.org,
        bhelgaas@google.com, kw@linux.com, lorenzo.pieralisi@arm.com,
        nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

On Thu, Nov 24, 2022 at 12:09 AM David E. Box
<david.e.box@linux.intel.com> wrote:
>
> Hi You-Sheng,
>
> On Wed, 2022-11-23 at 18:27 +0800, You-Sheng Yang wrote:
> > Hi David,
> >
> > On 11/3/22 10:18, David E. Box wrote:
> > > This series adds a work around for enabling PCIe ASPM and for setting PCIe
> > > LTR values on VMD reserved root ports on select platforms. While
> > > configuration of these capabilities is usually done by BIOS, on these
> > > platforms these capabilities will not be configured because the ports are
> > > not visible to BIOS. This was part of an initial design that expected the
> > > driver to completely handle the ports, including power management. However
> > > on Linux those ports are still managed by the PCIe core, which has the
> > > expectation that they adhere to device standards including BIOS
> > > configuration, leading to this problem.
> > >
> > > The target platforms are Tiger Lake, Alder Lake, and Raptor Lake though the
> > > latter has already implemented support for configuring the LTR values.
> > > Meteor Lake is expected add BIOS ASPM support, eliminating the future need
> > > for this work around.
> >
> >
> > It appears to me that this patch series works only on Tiger Lake. We
> > have tried to revert our current work-arounds in Ubuntu kernels
> > generic-5.15/oem-5.17/oem-6.0/unstable-6.1 and apply this series, the
> > prebuilt kernels can be found in:
> >
> >    https://launchpad.net/~vicamo/+archive/ubuntu/ppa-1996620
> >
> > However, only TGL can still enter PC10 as before.
> >
> >
> > ADL-M, RPL platforms will stay in PC3 with vmd LTR set, but ASPM
> > disabled.
>
> For the patch to work BIOS must allow the OS to control ASPM. If this is not the
> case then you will see the message "ACPI FADT declares the system doesn't
> support PCIe ASPM, so disable it". Please check for this on the systems that
> don't work. If so the only option is a BIOS change to enable it.

Thank you. It's exactly what you said. The ADL-M/RPL platforms I have
do not support OS PCIe ASPM.

> David
>
> >  i915 RC6 blocked, too:
> >
> > $ sudo cat /sys/kernel/debug/dri/
> >
> > 0/i915_dmc_info
> > ...
> > DC3CO count: 0
> > DC3 -> DC5 count: 100
> > DC5 -> DC6 count: 0
> >
> >
> > > Note, the driver programs the LTRs because BIOS would also normally do this
> > > for devices that do not set them by default. Without this, SoC power
> > > management would be blocked on those platform. This SoC specific value is
> > > the maximum latency required to allow the SoC to enter the deepest power
> > > state.
> > >
> > > This patch addresses the following open bugzillas on VMD enabled laptops
> > > that cannot enter low power states.
> > >
> > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=212355
> > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=215063
> > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=213717
> > >
> > > David E. Box (3):
> > >    PCI: vmd: Use PCI_VDEVICE in device list
> > >    PCI: vmd: Create feature grouping for client products
> > >    PCI: vmd: Add quirk to configure PCIe ASPM and LTR
> > >
> > > Michael Bottini (1):
> > >    PCI/ASPM: Add pci_enable_link_state()
> > >
> > >   drivers/pci/controller/vmd.c | 96 ++++++++++++++++++++++++++----------
> > >   drivers/pci/pcie/aspm.c      | 54 ++++++++++++++++++++
> > >   include/linux/pci.h          |  7 +++
> > >   3 files changed, 131 insertions(+), 26 deletions(-)
> > >
> > >
> > > base-commit: 247f34f7b80357943234f93f247a1ae6b6c3a740
> >
> >
> > Regards,
> > You-Sheng Yang
> >
>


-- 
Regards,
You-Sheng Yang

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-pci-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 8481CC352A1
	for <linux-pci@archiver.kernel.org>; Wed,  7 Dec 2022 09:05:47 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S229777AbiLGJFq (ORCPT <rfc822;linux-pci@archiver.kernel.org>);
        Wed, 7 Dec 2022 04:05:46 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54240 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S229515AbiLGJFp (ORCPT
        <rfc822;linux-pci@vger.kernel.org>); Wed, 7 Dec 2022 04:05:45 -0500
Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DB6526AC2;
        Wed,  7 Dec 2022 01:05:44 -0800 (PST)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
        by mailout.west.internal (Postfix) with ESMTP id B2C0F32009E1;
        Wed,  7 Dec 2022 04:05:42 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
  by compute3.internal (MEProxy); Wed, 07 Dec 2022 04:05:43 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ljones.dev; h=cc
        :cc:content-transfer-encoding:content-type:date:date:from:from
        :in-reply-to:in-reply-to:message-id:mime-version:references
        :reply-to:sender:subject:subject:to:to; s=fm3; t=1670403942; x=
        1670490342; bh=JFoXxMClww2l4RygoD+XW2GsjAAbe5uznYYD7nEj5+A=; b=A
        m+3Xd9pOm+9ctEko/ry4U6uTcBmcKTxoSXhrOYjFkNzs/UPFQ3eYMIW6MPgw2H6r
        W71NHDCfNlVOyxGK6QEtxyF8rdWdUjylzGuvSzy+iI1cTIyDvUZOhVgBzXAR4pdP
        zDSd0d9YPMphKkyku5QEjEG38WA26pdIxpRsHBm55ScFgaM4uCv5/xQ+rjaYprZL
        A+fvdT8D23+KOupjIeylZk5InsaqGNrVUifuJpVIM5mUkn0HoOyyvrRmtFXSq5Yl
        X1CHYaDxQcHEXMHnut+AsYWLGLPjewjytElsnMWmy1+yzdHdszmkfkCF+BE8Ba8Z
        d1Ny7AxdMcZGwGVoyRnaA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
        messagingengine.com; h=cc:cc:content-transfer-encoding
        :content-type:date:date:feedback-id:feedback-id:from:from
        :in-reply-to:in-reply-to:message-id:mime-version:references
        :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy
        :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1670403942; x=
        1670490342; bh=JFoXxMClww2l4RygoD+XW2GsjAAbe5uznYYD7nEj5+A=; b=o
        u/WWNWj8Jc92d/UtZbf4Nr+ZIPhRQ0768ZQ/Sf+INAMi26NBhtJLUu9aNRQpIK9I
        Dam5k6gOJvt4nfYzC9GRZMJosLhHrmL/Pm/8dRS05pKgo6R9gGptihDfIDu9dNYb
        HJvCMcVklYC9LtlK36zKlfetDEM/mF5y2GF8AGbWq1D7lUzoNiZJRbGbSlQDH9p5
        ACR3p5CiE0mnOoBA0RkzabymijwuKJmhCobq8TlPRt9je7WUgCesGxbKkEguZVG+
        l1xHMxvwWGFnyNSwJOpnHQtIhs6OMDxTKMQ74pAabh6fLRKqrS/w9WScvW5DyFCy
        4gxPK3Ki3pdCGhXaZWGPw==
X-ME-Sender: <xms:ZVeQY6JcmBl9r8UaSgPQlsL6kBpyH4upmsXGoJ4JZ0S4Apf8Uzw2tQ>
    <xme:ZVeQYyIZZ_J4e1TTTEmsSwTW5tyJbARUJ4hEG-TJP_Lki94M9bKzSpCOlPByJH2t7
    pMAV4LFMsEEUlVV95U>
X-ME-Received: <xmr:ZVeQY6uXMnU1qz7wIIZC_5ZP-dESYU4i7Aar3YknT1ZO2v4AkA5mOobw-mndir-YpJ-2wA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrudejgdduvdelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucenucfjughrpefkuffhvfevffgjfhgtgfgfggesth
    hqredttderjeenucfhrhhomhepnfhukhgvucflohhnvghsuceolhhukhgvsehljhhonhgv
    shdruggvvheqnecuggftrfgrthhtvghrnhepteegteefhefhteegleejudfffffghfekle
    eijeeugfffteeiudefvdetteeuuedvnecuffhomhgrihhnpehkvghrnhgvlhdrohhrghen
    ucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehluhhkvg
    eslhhjohhnvghsrdguvghv
X-ME-Proxy: <xmx:ZVeQY_ahVbtfiYjhg_k2eYSq-uPXq1xkiFLHac3vmlI3yXJqdoKeog>
    <xmx:ZVeQYxa-BJIz-lhlN77iSktQ8_NETM1TjQY8KXMwVRQe91NiGV1Q5w>
    <xmx:ZVeQY7BP-z-UuGNNFk8YZcB2ihklL1TIm-yrdTrk8nj3ewRVuCLcJA>
    <xmx:ZleQY-QBDw9ZgrfCGBIIbxE3z6H9QizXgV3ZnQTGdJ6RWbka1UWQVQ>
Feedback-ID: i5ec1447f:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 7 Dec 2022 04:05:35 -0500 (EST)
Message-ID: <8b8c7ae512291532ea227b8a1c3f87c460e11e2f.camel@ljones.dev>
Subject: Re: [PATCH V8 0/4] PCI: vmd: Enable PCIe ASPM and LTR on select
 hardware
From:   Luke Jones <luke@ljones.dev>
To:     "David E. Box" <david.e.box@linux.intel.com>,
        nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev,
        lorenzo.pieralisi@arm.com, hch@infradead.org, kw@linux.com,
        robh@kernel.org, bhelgaas@google.com, michael.a.bottini@intel.com,
        rafael@kernel.org, me@adhityamohan.in
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Date:   Wed, 07 Dec 2022 22:05:28 +1300
In-Reply-To: <20221103021822.308586-1-david.e.box@linux.intel.com>
References: <20221103021822.308586-1-david.e.box@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.46.2 (by Flathub.org) 
MIME-Version: 1.0
Precedence: bulk
List-ID: <linux-pci.vger.kernel.org>
X-Mailing-List: linux-pci@vger.kernel.org

On Wed, 2022-11-02 at 19:18 -0700, David E. Box wrote:
> This series adds a work around for enabling PCIe ASPM and for setting
> PCIe
> LTR values on VMD reserved root ports on select platforms. While
> configuration of these capabilities is usually done by BIOS, on these
> platforms these capabilities will not be configured because the ports
> are
> not visible to BIOS. This was part of an initial design that expected
> the
> driver to completely handle the ports, including power management.
> However
> on Linux those ports are still managed by the PCIe core, which has
> the
> expectation that they adhere to device standards including BIOS
> configuration, leading to this problem.
>=20
> The target platforms are Tiger Lake, Alder Lake, and Raptor Lake
> though the
> latter has already implemented support for configuring the LTR
> values.
> Meteor Lake is expected add BIOS ASPM support, eliminating the future
> need
> for this work around.
>=20
> Note, the driver programs the LTRs because BIOS would also normally
> do this
> for devices that do not set them by default. Without this, SoC power
> management would be blocked on those platform. This SoC specific
> value is
> the maximum latency required to allow the SoC to enter the deepest
> power
> state.
>=20
> This patch addresses the following open bugzillas on VMD enabled
> laptops
> that cannot enter low power states.
>=20
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D212355
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D215063
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D213717
>=20
> David E. Box (3):
> =C2=A0 PCI: vmd: Use PCI_VDEVICE in device list
> =C2=A0 PCI: vmd: Create feature grouping for client products
> =C2=A0 PCI: vmd: Add quirk to configure PCIe ASPM and LTR
>=20
> Michael Bottini (1):
> =C2=A0 PCI/ASPM: Add pci_enable_link_state()
>=20
> =C2=A0drivers/pci/controller/vmd.c | 96 ++++++++++++++++++++++++++-------=
-
> --
> =C2=A0drivers/pci/pcie/aspm.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 54 ++++++++=
++++++++++++
> =C2=A0include/linux/pci.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 |=C2=A0 7 +++
> =C2=A03 files changed, 131 insertions(+), 26 deletions(-)
>=20
>=20
> base-commit: 247f34f7b80357943234f93f247a1ae6b6c3a740

I'd like to confirm that this patch series solves the VMD power issues
present on the ASUS ROG M16 (GU603) laptop range (PCI dev ID =3D 467f).

The difference is quite drastic.