Allure 测试报告

🍰 一款轻量级的开源自动化测试报告生成框架,支持绝大部分测试框架,例如:TestNG、JUnit、PyTest、UnitTest等。

1 allure框架

  • allure框架
    • allure2下载allure-2.20.1.zip,JDK1.8版本,pip install allure-pytest安装Python依赖。
    • 添加变量:控制面板—>用户账户—>更改环境变量—>Path—>添加...\allure-2.20.1\bin
    • Python依赖包含allure-pytest和allure-python-commons两个包,生成与allure2兼容的报告数据。
1
2
3
4
5
6
7
8
9
10
11
12
13
import pytest                                                   # test_case.py


@pytest.mark.skip()
def test_a():
print("---use case a---")


def test_b():
print("---use case b---")

# pytest -vs --alluredir=./report/xml --clean-alluredir # 生成xml格式的报告数据
# allure serve report\xml # 直接在本地浏览器中打开报告
  • 报告结构介绍
    • Overview:总览。
    • Categories:用例类别,默认failed和error,可快速查看哪些用例执行是failed和error的。
    • Suites:即所有用例的层级关系,可以依据package、module、class和method来查找用例。
    • Graphs:测试结果的图形化展示,包括分布图等。
    • Timeline:测试用例的执行顺序,包括执行时间。
    • Behaviors:依据epic、feature、story进行用例分组。
    • Packages:按package、module进行测试用例的分组。

1-1 添加环境变量

  • 添加环境变量
    • Overview的ENVIRONMENT
      • 在生成xml格式的报告数据文件夹中,创建一个environment.properties文件。
      • 同理也可以在生成xml格式的报告数据文件夹中创建一个environment.xml文件。
    • 环境变量添加完成后,需要重新执行命令allure serve report\xml,在本地浏览器中重新打开报告才能生效。

(1) environment.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<environment>                                                   <!-- environment.xml文件 -->
<parameter> <!-- .xml配置优先于.properties -->
<key>Browser</key>
<value>Chrome</value>
</parameter>
<parameter>
<key>Browser.Version</key>
<value>100.0</value>
</parameter>
<parameter>
<key>Stand</key>
<value>Production</value>
</parameter>
<parameter>
<key>Type</key>
<value>xml</value>
</parameter>
</environment>

(2) environment.properties

1
2
3
4
Browser=Chrome                                                   # environment.properties文件
Browser.Version=107.0 # 最好不带注释信息,可能乱码显示
Stand=Production
Type=Properties

1-2 自定义缺陷分类

  • 自定义缺陷分类
    • Categories:在生成xml格式的报告数据文件夹中创建一个categories.json文件。
    • Json文件参数说明
      • name:必填,类别名称,可以是中文。
      • messageRegex:可选,用例运行的错误消息,默认".*",可以通过正则表达式进行匹配。
      • traceRegex:可选,用例运行错误消息的堆栈跟踪,默认".*",可以通过正则表达式匹配。
      • matchedStatuses:可选,用例运行状态,默认["failed","broken","passed","skipped","unknown"]
      • 如果用例运行结果的状态在列表中,并且错误消息和堆栈跟踪都与之模式相匹配,则测试结果属于该类别。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
[                                                               // categories.json文件
{ // 文件中最好不要带注释信息,否则无效
"name": "Ignored tests",
"matchedStatuses": ["skipped"]
},
{
"name": "Infrastructure problems",
"matchedStatuses": ["broken", "failed"],
"messageRegex": ".*bye-bye.*"
},
{
"name": "Outdated tests",
"matchedStatuses": ["broken"],
"traceRegex": ".*FileNotFoundException.*"
},
{
"name": "Execution failed",
"matchedStatuses": ["failed"]
},
{
"name": "Test defects",
"matchedStatuses": ["broken"]
}
]

2 allure注释说明

  • allure注释说明
    • @allure.step():测试用例的步骤,参数传什么,在allure中步骤名就显示什么。
    • @allure.attach():显示不同类型的附件,补充测试结果,可依据自身情况调整。
    • @allure.title():自定义用例标题,支持占位符传递关键字参数,标题更具可读性。
    • @allure.description():为测试用例添加详细的描述,并且将其展示到测试报告中。
    • 访问链接:将测试报告与bug管理工具或测试管理系统进行关联。
      • @allure.link():访问网址的链接。
      • @allure.issue():访问缺陷的链接。
      • @allure.testcase():访问用例链接。
    • BDD风格的标记:行为驱动开发(BDD),在测试驱动开发(TDD)基础上发展而来的一种软件开发方法。
      • @allure.epic():相当于module级的标签,敏捷里面的概念。
      • @allure.feature():相当于class级的标签,epic是feature的父级。
      • @allure.story():相当于method级的标签,feature是story的父级。
    • @allure.severity():划分测试用例的等级,并展示到测试报告内。
    • 测试套件:已有BDD风格的标记,测试套件不常用,可以做简单了解。
      • @allure.suite():是与当前测试套件同级的测试套件。
      • @allure.sub_suite():当前测试套件的子级测试套件。
      • @allure.parent_suite():当前测试套件的父级测试套件。

2-1 用例步骤

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import allure                                                   # test_step.py


@allure.step("Open the login site.")
def open():
pass


@allure.step("Enter your account and password.")
def input_username_password():
input_username_password_and_login("admin", "123456")


@allure.step("Enter account, password {arg1}, {arg2}, and click login.")
def input_username_password_and_login(arg1, arg2):
pass


@allure.step("Verify the login process.")
def test_login():
open()
input_username_password()

# pytest -vs test_step.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

2-2 用例补充

  • 用例补充
    • allure.attach(body, name, attachment_type, extension):用于未想要导入现成的附件情况。
    • allure.attach.file(source, name, attachment_type, extension):用于已有对应的附件情况。
      • body(显示内容)、name(附件名)、source(附件路径)。
      • extension(附件扩展名)、attachment_type(附件类型)。
    • 附件类型
      • text
        • TSV = ("text/tab-separated-values", "tsv")
        • HTML = ("text/html", "html")CSV = ("text/csv", "csv")
        • TEXT = ("text/plain", "txt")URI_LIST = ("text/uri-list", "uri")
      • image
        • PNG = ("image/png", "png")BMP = ("image/bmp", "bmp")
        • JPG = ("image/jpg", "jpg")GIF = ("image/gif", "gif")
        • TIFF = ("image/tiff", "tiff")SVG = ("image/svg-xml", "svg")
      • video
        • MP4 = ("video/mp4", "mp4")
        • OGG = ("video/ogg", "ogg")
        • WEBM = ("video/webm", "webm")
      • application
        • PCAP = ("application/vnd.tcpdump.pcap", "pcap")
        • XML = ("application/xml", "xml")JSON = ("application/json", "json")
        • PDF = ("application/pdf", "pdf")YAML = ("application/yaml", "yaml")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import allure                                                   # test_attach.py


@allure.step("Open the login site.")
def open():
pass


@allure.step("Enter your account and password.")
def input_username_password():
input_username_password_and_login("admin", "123456")


@allure.step("Enter account, password {arg1}, {arg2}, and click login.")
def input_username_password_and_login(arg1, arg2):
pass


@allure.step("Verify the login process.")
def test_login():
open()
input_username_password()


def test_attach():
allure.attach("<head></head><body><strong>Add an html page.</strong></body>",
"Attach with HTML type", allure.attachment_type.HTML)
# 注意:需要在当前test_attach.py文件的目录下放置report.html和report.jpg,否则报错
allure.attach.file("./report.html", attachment_type=allure.attachment_type.HTML)
allure.attach.file("./report.jpg", attachment_type=allure.attachment_type.JPG)

# pytest test_attach.py --alluredir=./report/annex
# allure serve ./report/annex

2-3 用例标题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import pytest                                                   # test_title.py
import allure


@allure.title("Pre-operation: Get username.")
@pytest.fixture()
def get_use(request):
use = request.param
print(f"Username: {use}")
return use


@allure.title("Pre-operation: Get password.")
@pytest.fixture()
def get_pwd(request):
pwd = request.param
print(f"Password: {pwd}")
return pwd


use_data = ["Holy", "Lucy"]
pwd_data = ["1235", "1236"]


@allure.title("Username: {get_use} --- Password: {get_pwd}.") # 占位符传递关键字参数
@pytest.mark.parametrize("get_use", use_data, indirect=True) # 动态标题,结合parametrize使用
@pytest.mark.parametrize("get_pwd", pwd_data, indirect=True)
def test_get_userinfo(get_use, get_pwd):
print(f"Username: {get_use} --- Password: {get_pwd}.")

# pytest -vs test_title.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

2-4 用例描述

  • 用例描述
    • 在用例函数声明的下方使用""" """,使用@allure.description()装饰器。
    • 使用@allure.description_html()装饰器添加html描述,类似attach传html。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import allure                                                   # test_description.py


def test_description1():
"""
Verify 1=1
:return:
"""
assert 1 == 1


@allure.description("Verify 1=1")
def test_description2():
assert 1 == 1


@allure.description_html("""<h1>Verify 1=1</h1>""")
def test_description3():
assert 1 == 1

# pytest -vs test_description.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

2-5 访问链接

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import allure                                                   # test_access.py

test_case_link = "https://docs.qq.com/mind/DSnRoQU9kRlNTRm1v"


@allure.link("https://stitch-top.github.io")
def test_link():
pass


@allure.link("https://stitch-top.github.io", name="Click on the title to view the content.")
def test_name_link():
pass


@allure.issue("https://docs.qq.com/mind/DSkNVTGVwVUhzYnBt", "Defect link")
def test_issue_link():
pass


@allure.testcase(test_case_link, "Test case link")
def test_testcase_link():
pass

# pytest -vs test_access.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

2-6 BDD风格的标记

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import allure                                                   # test_style.py


@allure.story("epic_a")
def test_a():
pass


@allure.story("story_a")
def test_b():
pass


@allure.story("story_b")
def test_c():
pass


@allure.feature("feature_b")
@allure.story("story_b")
def test_d():
pass


@allure.epic("epic_b")
@allure.feature("feature_b")
@allure.story("story_b")
def test_e():
pass

# pytest -vs test_style.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

# 只运行epic中名为epic_b的用例
# pytest --alluredir ./report/allure --allure-epics=epic_b

# 只运行feature中名为feature_b的用例
# pytest --alluredir ./report/allure --allure-features=feature_b

# 只运行story_a,story_b的用例
# pytest test_style.py --allure-stories story_a,story_b

# pytest test_style.py --allure-features feature_b --allure-stories story_a

2-7 用例等级的划分

  • 用例等级的划分
    • blocker:阻塞缺陷,功能未实现。
    • critical:严重缺陷,功能点缺失。
    • normal:一般缺陷,边界情况,格式错误。
    • minor:次要缺陷,界面错误及UI需求不符。
    • trivial:轻微缺陷,必须项无提示,或提示不规范。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
import allure                                                   # test_severity.py


def test_a(): # 没标记severity的方法默认优先级为normal
pass


@allure.severity(allure.severity_level.BLOCKER)
def test_b(): # 优先级为blocker
pass


@allure.severity(allure.severity_level.CRITICAL)
def test_c(): # 优先级为critical
pass


@allure.severity(allure.severity_level.NORMAL)
def test_d(): # 优先级为normal
pass


@allure.severity(allure.severity_level.MINOR)
def test_e(): # 优先级为minor
pass


@allure.severity(allure.severity_level.TRIVIAL)
def test_f(): # 优先级为trivial
pass


@allure.severity(allure.severity_level.NORMAL)
class TestCase(object): # 类优先级normal
def test_g(self): # 默认继承类优先级normal
pass

@allure.severity(allure.severity_level.CRITICAL)
def test_h(self): # 类优先级normal,函数优先级critical
pass

# pytest -vs test_severity.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml
# pytest test_severity.py --allure-severities=normal,critical # 只运行优先级为normal和critical的用例
# pytest test_severity.py --allure-severities blocker,critical # 只运行优先级为blocker和critical的用例

2-8 测试套件的使用

  • 测试套件的使用
    • 未被suite修饰时,一个py文件和一个class默认就是一个测试套件。
    • 关系为py包含class套件,py和class下都可有它们各自的测试用例。
    • 被suite修饰的类和方法,会被认为是和当前文件同级的测试套件。
    • 如果类和类内方法同时被修饰,类修饰会被类内方法的修饰替代。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import allure                                                   # test_suite.py


@allure.suite("test kit: suite")
def test_a():
pass


@allure.suite("test kit: sub_suite")
def test_b():
pass


@allure.suite("test kit: parent_suite")
def test_c():
pass


@allure.suite("test suites")
class TestSuiteX:
@allure.suite("class internal test kit d")
def test_d(self):
pass

def test_e(self):
pass


class TestSuiteY: # 无suite修饰的测试类
@allure.suite("class internal test kit f")
def test_f(self):
pass

def test_g(self):
pass

# pytest -vs test_suite.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

3 命令行参数介绍

  • 命令行参数介绍
    • pytest --alluredir=./report/xml test_file.py:执行test_file.py文件用例。
      • 当多次执行不同文件用例时,报告存放的路径保持一致,那么报告数据将会累加。
      • 使用--clean-alluredir命令行参数生成的报告,可以清空报告的历史执行记录。
      • 命令:pytest --alluredir=./report/xml test_file.py --clean-alluredir
    • allure -h:查看参数帮助文档。
      • generate:生成allure的html报告。
      • serve:启动allure服务,打开报告。
      • open:打开用generate生成的报告。
    • 浏览器打开allure报告的两种方式
      • pytest -sq --alluredir=./report/xml:执行测试用例文件,并指定结果目录。
      • allure serve ./report/xml:打开allure测试报告。
      • allure generate -c -o ./report/xml ./report/html:生成allure的html报告。
      • allure open ./report/html:打开allure测试报告。

4 用例标题动态化

  • 用例标题动态化
    • 使用参数化,报告默认的用例标题为用例名称,可读性差。
    • 参数化使用ids,会保留用例名称,也无法完全解决可读性差的问题。
    • 若使用@allure.title()装饰器写死用例标题,后期维护成本较高。
    • 将parametrize参数化数据驱动与@allure.title()装饰器结合使用,更加高效便捷。

4-1 参数化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import pytest                                                   # test_param1.py

info = [
{"username": "admin1", "pwd": "12345"},
{"username": "admin2", "pwd": "12346"},
{"username": "admin3", "pwd": "12347"}
]


@pytest.fixture()
def login(request):
param = request.param
print(f"The account number is: {param['username']} and the password is: {param['pwd']}.")
return {"code": 0, "msg": "success!"}


@pytest.mark.parametrize("login", info, indirect=True) # 参数化默认的标题
def test_login(login):
assert login["code"] == 0

# pytest -vs test_param1.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

4-2 参数化id

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import pytest                                                   # test_param2.py

info = [
pytest.param({"username": "admin1", "pwd": "12345"}, id="admin1 login success"),
pytest.param({"username": "admin2", "pwd": "12346"}, id="admin2 failed to log in"),
pytest.param({"username": "admin3", "pwd": "12347"}, id="admin3 username does not exist")
]


@pytest.fixture()
def login(request):
param = request.param
print(f"The account number is: {param['username']} and the password is: {param['pwd']}.")
return {"code": 0, "msg": "success!"}


@pytest.mark.parametrize("login", info, indirect=True) # 参数化ids
def test_login(login):
assert login["code"] == 0

# pytest -vs test_param2.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

4-3 参数化ids

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import pytest                                                   # test_param3.py

info = [
{"username": "admin1", "pwd": "12345"},
{"username": "admin2", "pwd": "12346"},
{"username": "admin3", "pwd": "12347"}
]

ids = ["admin1 login success", "admin2 failed to log in", "admin3 username does not exist"]


@pytest.fixture()
def login(request):
param = request.param
print(f"The account number is: {param['username']} and the password is: {param['pwd']}.")
return {"code": 0, "msg": "success!"}


@pytest.mark.parametrize("login", info, ids=ids, indirect=True)
def test_login(login): # 参数化ids
assert login["code"] == 0

# pytest -vs test_param3.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

4-4 固定标题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import pytest                                                   # test_param4.py
import allure

info = [
{"username": "admin1", "pwd": "12345"},
{"username": "admin2", "pwd": "12346"},
{"username": "admin3", "pwd": "12347"}
]


@pytest.fixture()
def login(request):
param = request.param
print(f"The account number is: {param['username']} and the password is: {param['pwd']}.")
return {"code": 0, "msg": "success!"}


@allure.title("Login use case: admin1 success, admin2 failure, admin3 user name does not exist.")
@pytest.mark.parametrize("login", info, indirect=True) # @allure.title()固定标题
def test_login(login):
assert login["code"] == 0

# pytest -vs test_param4.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

4-5 参数化与装饰器结合

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
import pytest                                                   # test_param5.py
import allure

info = [
{"username": "admin1", "pwd": "12345"},
{"username": "admin2", "pwd": "12346"},
{"username": "admin3", "pwd": "12347"}
]


@pytest.fixture()
def login(request):
param = request.param
print(f"The account number is: {param['username']} and the password is: {param['pwd']}.")
return {"code": 0, "msg": "success!"}


@allure.title("Login use case: {login}.") # 参数化与title动态生成标题,结合fixture
@pytest.mark.parametrize("login", info, indirect=True)
def test_login(login):
assert login["code"] == 0


"""
info = [
("admin1", "12345", "admin1 login success"),
("admin2", "12346", "admin2 failed to log in"),
("admin3", "12347", "admin3 username does not exist")
]


@allure.title("Login use case: {title}.") # 参数化与title动态生成标题,不结合fixture
@pytest.mark.parametrize("username, pwd, title", info)
def test_login(username, pwd, title):
pass
"""

# pytest -vs test_param5.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

5 dynamic动态标记

  • dynamic动态标记
    • allure.dynamic.issueallure.dynamic.linkallure.dynamic.testcase
    • allure.dynamic.storyallure.dynamic.featureallure.dynamic.description
    • allure.dynamic.titleallure.dynamic.severityallure.dynamic.description_html

5-1 更改标题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import allure                                                   # test_dynamic1.py


@allure.title("Title: Use case will be replaced after successful execution.")
def test_with_dynamic_title_success():
assert 1 + 1 == 2
allure.dynamic.title("Success: updated title.")


@allure.title("Title: Use case execution will not be replaced after failure.")
def test_with_dynamic_title_failed():
assert 1 + 1 == 3
allure.dynamic.title("Failure: Invalid title.") # Python执行机制影响,该行代码不会执行

# pytest -vs test_dynamic1.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

5-2 添加描述

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import allure                                                   # test_dynamic2.py


def test_a():
"""
Description: The use case will be replaced after successful execution.
:return:
"""
assert 1 + 1 == 2
allure.dynamic.description("Success: updated description.")


def test_b():
"""
Description: The use case will not be replaced after a failed execution.
:return:
"""
assert 1 + 1 == 3
allure.dynamic.description("Failure: Invalid description.")

# pytest -vs test_dynamic2.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

5-3 参数化设置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import allure                                                   # test_dynamic3.py
import pytest

info = [
["admin1", "12345", "Use case login success."], # 这里必须是列表,字典顺序会乱
["admin2", "12346", "Use case login failed."],
["admin3", "12347", "Account does not exist."]
]


@pytest.mark.parametrize("username, pwd, title", info)
def test_login(username, pwd, title):
"""
:param username:
:param pwd:
:param title:
:return:
"""
allure.dynamic.title(title)
allure.dynamic.description(f"Username: {username}, Password: {pwd}")

# pytest -vs test_dynamic3.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

5-4 结合其他特性

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import allure                                                   # test_dynamic4.py


def test_case():
assert 1 == 1
allure.dynamic.feature("Dynamic feature")
allure.dynamic.story("Dynamic story")
allure.dynamic.severity(allure.severity_level.BLOCKER)
allure.dynamic.link("Dynamic link: https://stitch-top.github.io")
allure.dynamic.issue("Dynamic issue: https://docs.qq.com/mind/DSkNVTGVwVUhzYnBt")
allure.dynamic.testcase("Dynamic testcase: https://docs.qq.com/mind/DSnRoQU9kRlNTRm1v")

# pytest -vs test_dynamic4.py --alluredir=./report/xml --clean-alluredir
# allure serve report\xml

6 pytest-xdist插件

  • pytest-xdist插件
    • 为了让自动化测试用例可分布式执行,节省时间,引入pytest-xdist插件。
    • pytest-xdist属于进程级别的并发,分布式执行测试用例的设计原则如下。
      • 用例之间相互独立:保证用例之间没有依赖关系,可以完全独立运行。
      • 用例没有执行顺序:保证随机顺序的执行测试用例,都可以正常执行。
      • 用例之间互不影响:保证用例的运行结果不会影响到其他的测试用例。
    • 命令窗口下安装pytest-xdist插件:pip install pytest-xdist
    • 原理
      • xdist通过产生一个或多个由master控制的workers来工作,再按master的指示运行用例。
      • 每个workers负责执行收集到的完整测试用例集,master本身是不执行任何测试用例的。
    • 命令
      • 指定CPU的具体个数:pytest -s -n num-n可以用--numprocesses代替。
      • 使用与当前计算机CPU内核一样多的进程,速度提升大:pytest -s -n auto
1
2
3
4
5
6
7
8
9
/:                                                               # 项目结构
- conftest.py
- test_case.py
test_module1:
- test_case1.py
test_module2:
- test_case2.py
test_module3:
- test_case3.py

6-1 conftest.py

1
2
3
4
5
6
7
8
9
10
import pytest                                                   # conftest.py


@pytest.fixture(scope="session")
def login():
print("---start use case---")
name, pwd = "admin", "12345"

yield name, pwd
print("---exit use case---")

6-2 test_case.py

1
2
3
4
5
6
7
8
import pytest                                                    # test_case.py
from time import sleep


@pytest.mark.parametrize("x", list(range(10)))
def test_index(login, x):
sleep(1)
print("---home use case---", x)

6-3 test_case1.py

1
2
3
4
5
6
7
8
import pytest                                                    # test_case1.py
from time import sleep


@pytest.mark.parametrize("x", list(range(10)))
def test_case1(x):
sleep(1)
print("module1: test case 1", x)

6-4 test_case2.py

1
2
3
4
5
6
7
8
import pytest                                                    # test_case2.py
from time import sleep


@pytest.mark.parametrize("x", list(range(10)))
def test_case2(x):
sleep(1)
print("module2: test case 2", x)

6-5 test_case3.py

1
2
3
4
5
6
from time import sleep                                           # test_case3.py


def test_case3():
sleep(1)
print("module3: test case 3")

6-6 两种执行方式

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
> pytest                                                        # 不使用分布式执行
=================================== test session starts ===================================
...
collected 31 items

test_case.py .......... [ 32%]
test_module1\test_case1.py .......... [ 64%]
test_module2\test_case2.py .......... [ 96%]
test_module3\test_case3.py .......... [100%]
=================================== 31 passed in 31.43s ===================================

> pytest -s -n auto # 使用分布式执行
=================================== test session starts ===================================
...
gw0 [31] / gw1 [31] / gw2 [31] / gw3 [31] / gw4 [31] / gw5 [31]
...............................
=================================== 31 passed in 7.45s ====================================

7 插件的文件锁问题

  • 插件的文件锁问题
    • 优点
      • xdist插件充分利用机器多核CPU的优势,提高了资源利用率。
      • 将常用的功能放到fixture,更是提高了代码的复用性和维护性。
    • 缺点
      • 接口自动化时,通常会将接口放到fixture中,并设置scope=session,全局只运行一次。
      • 此时若使用xdist插件进行分布式执行,scope=session的fixture将无法保证只运行一次。
    • 尽管没有内置支持确保scope=session的fixture只执行一次,但可通过文件锁进行进程间的通信来实现。
      • 使用xdist指定n个线程执行用例时,使用文件锁,fixture实际只执行一次。
      • 其他n-1个进程通过进程间读取共享缓存文件的方式,复用fixture的数据。
1
2
3
4
5
/:                                                               # 项目结构
- conftest.py
- test_case1.py
- test_case2.py
- test_case3.py

7-1 未用文件锁

1
2
3
4
5
6
7
8
import pytest                                                    # contest.py
import random


@pytest.fixture(scope="session") # 未用文件锁
def xdist_fixture():
data = str(random.random())
return data

7-2 使用文件锁

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
import os                                                       # contest.py
import json
import pytest
import random
from filelock import FileLock


@pytest.fixture(scope="session") # 使用文件锁
def xdist_fixture(tmp_path_factory, worker_id):
# 如果是单机运行,则运行这里的代码块【不可删除、修改】
if worker_id == "master":
"""
【自定义代码块】
这里写本身要做的操作,例如:登录请求、新增数据、清空数据库历史数据等
"""
data = str(random.random())
print(f"Master is executed for the first time, the data is {data}.")
os.environ["data"] = data
# 如果测试用例有需要,可以返回对应的数据,比如data
return data

# 如果是分布式运行,获取所有子节点共享的临时目录,无需修改【不可删除、修改】
root_tmp_dir = tmp_path_factory.getbasetemp().parent
fn = root_tmp_dir / "data.json" # 【不可删除、修改】

with FileLock(str(fn) + ".lock"): # 【不可删除、修改】
if fn.is_file(): # 【不可删除、修改】
# 缓存文件中读取数据,像登录操作的话就是data【不可删除、修改】
data = json.loads(fn.read_text())
print(f"The worker reads the cache file and the data is {data}.")
else:
"""
【自定义代码块】
这里写本身要做的操作,例如:登录请求、新增数据、清空数据库历史数据等
"""
data = str(random.random())
fn.write_text(json.dumps(data)) # 【不可删除、修改】
# If required by the test case, the corresponding data, such as token, can be returned.
print(f"The worker reads the cache file and the data is {data}.")

# 最好将后续需要保留的数据存在某个位置,比如os的环境变量中
os.environ["data"] = data

return data

7-3 test_case1.py

1
2
def test_case1(xdist_fixture):                                   # test_case1.py
print(f"Data1:{xdist_fixture}")

7-4 test_case2.py

1
2
def test_case2(xdist_fixture):                                   # test_case2.py
print(f"Data2:{xdist_fixture}")

7-5 test_case3.py

1
2
def test_case3(xdist_fixture):                                   # test_case3.py
print(f"Data3:{xdist_fixture}")

7-6 执行情况分析

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
> pytest -n 3 --alluredir=./report/xml --clean-alluredir        # 未用文件锁的执行情况
> allure serve report\xml
test_case1---Data10.8613196731229967 # session的fixture执行了3次
test_case2---Data20.2953396024322359
test_case3---Data30.5252796017515756

> pytest -n 3 --alluredir=./report/xml --clean-alluredir # 使用文件锁,xdist分布式执行
> allure serve report\xml
The worker reads the cache file and the data is 0.14799648172518787.
Data1:0.14799648172518787
The worker reads the cache file and the data is 0.14799648172518787.
Data2:0.14799648172518787
The worker reads the cache file and the data is 0.14799648172518787.
Data3:0.14799648172518787

> pytest --alluredir=./report/xml --clean-alluredir # 使用文件锁,不使用分布式执行
> allure serve report\xml
Master is executed for the first time, the data is 0.5299300518971257.
Data1:0.5299300518971257
Data2:0.5299300518971257
Data3:0.5299300518971257

8 重复执行用例插件

  • 重复执行用例插件
    • pytest-repeat是一个pytest插件,安装命令:pip install pytest-repeat
    • 可以重复执行单个测试用例,或多个测试用例,也可以自定义重复执行的次数。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
def test_a():                                                   # test_repeat1.py
print("---use case a---")


def test_b():
print("---use case b---")


# pytest -vs --count 3 test_repeat1.py # 指定重复运行3次
"""
=================================== test session starts ===================================
...
collected 6 items

test_repeat1.py::test_a[1-3] ---use case a---
PASSED
test_repeat1.py::test_a[2-3] ---use case a---
PASSED
test_repeat1.py::test_a[3-3] ---use case a---
PASSED
test_repeat1.py::test_b[1-3] ---use case b---
PASSED
test_repeat1.py::test_b[2-3] ---use case b---
PASSED
test_repeat1.py::test_b[3-3] ---use case b---
PASSED
==================================== warnings summary =====================================
...DeprecationWarning: Using or importing the ABCs from 'collections' instead of from
'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
return isinstance(x, collections.Callable)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=============================== 6 passed, 1 warning in 0.04s ==============================
"""

8-1 重复直到失败

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import random                                                   # test_repeat2.py


def test_case():
num = random.randint(1, 11)
assert num != 10


# pytest --count 11 -x test_repeat2.py # 重复执行11次直到失败停止
"""
=================================== test session starts ===================================
...
collected 11 items

test_repeat2.py ......F
======================================== FAILURES =========================================
_____________________________________ test_case[7-11] _____________________________________

def test_case():
num = random.randint(1, 11)
> assert num != 10
E assert 10 != 10

test_repeat2.py:6: AssertionError
==================================== warnings summary =====================================
...DeprecationWarning: Using or importing the ABCs from 'collections' instead of from
'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
return isinstance(x, collections.Callable)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================================== short test summary info ================================
FAILED test_repeat2.py::test_case[7-11] - assert 10 != 10
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================== 1 failed, 6 passed, 1 warning in 0.57s ========================
"""

8-2 repeat装饰器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import pytest                                                   # test_repeat3.py


@pytest.mark.repeat(3)
def test_case():
print("---use case---")


# pytest test_repeat3.py
"""
=================================== test session starts ===================================
...
collected 3 items

test_repeat3.py ... [100%]
==================================== warnings summary =====================================
...DeprecationWarning: Using or importing the ABCs from 'collections' instead of from
'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
return isinstance(x, collections.Callable)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=============================== 3 passed, 1 warning in 0.12s ==============================
"""

8-3 命令参数说明

  • 命令参数说明
    • --repeat-scope:覆盖默认的用例执行顺序,类似于fixture的scope。
    • function:默认,范围针对每个用例重复执行,然后再执行下一个用例。
    • class:以类为用例的集合单位,重复执行类里面的用例,再执行下一个用例。
    • module:以模块为集合单位,重复执行模块里面的用例,再执行下一个用例。
    • session:重复整个测试会话,即所有测试用例都执行一次,然后再执行第二次。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
def test_a():                                                   # test_repeat4.py
print("---use case a---")


def test_b():
print("---use case b---")


class TestCase1:
def test_c(self):
print("---use case c---")


class TestCase2:
def test_d(self):
print("---use case d---")


"""
# pytest -s --count=2 --repeat-scope=class test_repeat4.py # 重复执行class级别用例
.---use case a---
.---use case b---
.---use case a---
.---use case b---
.---use case c---
.---use case c---
.---use case d---
.---use case d---

# pytest -s --count=2 --repeat-scope=module test_repeat4.py # 重复执行module级别用例
.---use case a---
.---use case b---
.---use case c---
.---use case d---
.---use case a---
.---use case b---
.---use case c---
.---use case d---
"""

Allure 测试报告
https://stitch-top.github.io/2023/05/01/ce-shi-kuang-jia/tf03-allure/tf01-allure-ce-shi-bao-gao/
作者
Dr.626
发布于
2023年5月1日 21:30:55
许可协议