How Strong Is Windsurf in Writing Test Cases?

From the perspective of functional delivery, relying on AI to generate code from natural language is quite a helpless endeavor. As for the computer as a running medium, it currently cannot directly execute human natural language; to make the function effective, it must rely on code as an intermediate medium for escaping. However, a more ideal approach would be to completely abandon the intermediate medium of computer instructions and allow computers to directly execute natural language, although the current technological level cannot achieve this goal.
Although direct delivery is currently difficult, it is feasible in certain specific scenarios, such as testing acceptance. Any function provided for human use will be accepted by humans after delivery. Since the requirements are written in natural language by humans, and the delivered functions are accepted by humans, theoretically, AI can completely act as a substitute intermediate medium, converting requirements into acceptance criteria.
The main workflow for current testing acceptance is to generate test cases and execution plans based on requirements, testing specifications, methodologies, and personal experience, to test and accept the deliverables. Test cases come from requirements, which are written by humans, and the users of test cases are also humans. It can be said that both the input and output of test cases are natural language, which is precisely the field in which large language models excel.
So, what is the application effect of AI in this field? Does it really help actual work? With this question in mind, I tried using domestic tools like Doubao, Zhipu Qingyan, and Windsurf to generate test cases based on requirements. In practice, I was too lazy to write requirements, so I used the same prompt words to let them generate user stories on their own.
From the experience, Zhipu Qingyan can generate user stories, with scenarios including business functions and network security-related content, and the generated test cases are relatively complete. Windsurf, on the other hand, performs more professionally, and the experience is also more outstanding, making one marvel at this all-rounder who is hindered by coding capabilities.
Windsurf directly generated two markdown files: order_user_stories.md and order_test_cases.md, making it more convenient to use. In the user stories, its functional and non-functional scenarios are richer, for example, the network security scenarios are more abundant, while Zhipu did not mention transaction security scenarios; it even covers personal data protection-related content. In the test cases, its case descriptions are clearer, and the scenarios are more complete, such as including cases of high-concurrency ordering and data storage security. It can be said that in terms of professionalism and user experience, Windsurf is closer to the actual application level.
It should be noted that this is only a small-scale attempt, and there are many inadequacies in the operation process; the conclusions are for reference only.
  • Here are the detailed process information:
Prompt 1: You are an outstanding product manager capable of writing excellent user stories for e-commerce systems. Please simulate mainstream e-commerce websites like JD.com to write user stories for placing orders. Prompt 2: Please add functions related to network security.
  • Zhipu Qingyan
User Story: User Places Order (Adding Network Security Related Functions) As an online shopper, I want my personal information and transaction security to be protected while enjoying a convenient shopping experience, to enhance my trust in the e-commerce platform. Role: Shopper Purpose: To safely complete the purchase of goods Preconditions: The user has logged into the JD.com account. The user has browsed the product details and decided to purchase. The product inventory is sufficient. The e-commerce platform has implemented network security measures. Main Process: Add Product to Cart As a user, I can click the "Add to Cart" button on the product details page to add the product to my cart. The system should immediately respond, showing that the product has been successfully added to the cart and updating the number of products in the cart icon. The system should ensure that the operation of adding products to the cart is conducted through a secure connection. View Cart As a user, I can choose to continue shopping or click the cart icon to view the products in the cart. The system should display the cart page, listing all added products, including product name, price, quantity, and total price. The system should implement HTTPS encryption on the cart page to protect user data from being intercepted. Confirm Order Information As a user, I can click the "Proceed to Checkout" button on the cart page to enter the order confirmation page. The system should display the order confirmation page, including product list, shipping address, delivery method, payment method, and other information. The system should implement security measures such as preventing SQL injection on the order confirmation page to prevent data leakage. Choose Shipping Address As a user, I can choose an existing shipping address or add a new shipping address. The system should allow users to edit and save new shipping address information. The system should validate the user-input shipping address information to prevent malicious code injection. Choose Delivery Method As a user, I can choose a delivery method (such as standard express, JD express, etc.) according to my needs. The system should update the delivery cost and estimated delivery time based on the user's choice. The system should ensure that the process of selecting delivery methods is not affected by cross-site scripting attacks (XSS). Choose Payment Method As a user, I can choose one of the provided payment methods (such as online payment, cash on delivery, etc.). The system should ensure the diversity and security of payment methods. The system should use a secure payment gateway and implement additional security measures such as two-factor authentication (2FA). Submit Order As a user, after confirming that all information is correct, I can click the "Submit Order" button. The system should process the order request and display a successful order submission page after success, providing order details. The system should validate the order submission request to prevent duplicate submissions and order forgery. Successful Order Submission As a user, I want to see confirmation information for successful order submission, including order number and order status. The system should provide options for viewing order details and guide the user to make payment. The system should provide security tips to prevent phishing on the successful order page. Network Security Related Acceptance Criteria: All sensitive user operations must be transmitted via encrypted HTTPS. The system should strictly validate and sanitize user inputs to prevent SQL injection, XSS attacks, and other network security threats. The system should implement a two-factor authentication mechanism, especially during payment. The system should conduct regular security audits and vulnerability scans to ensure timely detection and remediation of security vulnerabilities. The user interface should include security tips to educate users on recognizing and preventing phishing and other fraudulent activities. The system should log all security-related events for auditing and incident investigation.
How Strong Is Windsurf in Writing Test Cases?
Figure 1. Zhipu Qingyan User Story
Below are the test cases generated based on the provided user story content: Test Case: User Securely Places Order Test Case 1: Add Product to Cart Goal: Verify whether the user can safely add a product to the cart. Preconditions: The user is logged in. The product details page has loaded. Steps: The user clicks the "Add to Cart" button on the product details page. The user observes the page feedback. Expected Result: The product is successfully added to the cart, and a prompt message is displayed. The number of products on the cart icon is updated. The browser shows a secure connection (HTTPS). Test Case 2: View Cart Goal: Verify whether the cart page securely displays product information. Preconditions: The product has been added to the cart. Steps: The user clicks the cart icon to view the cart. Expected Result: The cart page loads via HTTPS encryption. The page lists all product information, including name, price, quantity, and total price. Test Case 3: Confirm Order Information Goal: Verify the security and information display of the order confirmation page. Preconditions: The user has entered the cart page. Steps: The user clicks the "Proceed to Checkout" button. Expected Result: The order confirmation page loads, displaying product list, shipping address, delivery method, payment method, and other information. The page implements security measures to prevent SQL injection and other attacks. Test Case 4: Choose Shipping Address Goal: Verify whether the user can safely choose and edit the shipping address. Preconditions: The user has entered the order confirmation page. Steps: The user selects or adds a new shipping address. The user saves the edited address information. Expected Result: The system allows editing and saving address information. The system validates the address information to prevent malicious code injection. Test Case 5: Choose Delivery Method Goal: Verify the security of the delivery method selection process. Preconditions: The user has entered the order confirmation page. Steps: The user selects a delivery method. Expected Result: The system updates the delivery cost and estimated delivery time. The selection process is not affected by XSS attacks. Test Case 6: Choose Payment Method Goal: Verify the security and diversity of payment method selection. Preconditions: The user has entered the order confirmation page. Steps: The user selects a payment method. Expected Result: The system provides multiple payment options. The payment gateway is secure, implementing additional security measures such as two-factor authentication. Test Case 7: Submit Order Goal: Verify the security of the order submission process. Preconditions: The user has completed filling in all order information. Steps: The user clicks the "Submit Order" button. Expected Result: The system processes the order request, displaying a successful submission page. The system validates the request to prevent duplicate submissions and forgery. Test Case 8: Successful Order Submission Goal: Verify the security and information display of the order submission success page. Preconditions: The user has submitted the order. Steps: The user views the order submission success page. Expected Result: The page displays the order number and order status. Provides options for viewing order details. The page includes security tips to prevent phishing attacks. Network Security Related Acceptance Criteria Test Goals: Verify the implementation of network security measures. Steps: Check whether all sensitive operations are transmitted via HTTPS. Attempt SQL injection, XSS attacks, etc., to validate the system's protection capabilities. Validate the two-factor authentication mechanism during payment. Check whether the system conducts regular security audits and vulnerability scans. Observe whether the user interface includes security tips. Check whether the system records all security-related events.
How Strong Is Windsurf in Writing Test Cases?
Figure 2. Zhipu Qingyan Test Case
  • Windsurf
# E-commerce System User Order Process User Story ## 1. One-click Order from Cart **As** a registered user **I want** to place an order for the products in my cart with one click **so that** I can quickly complete the purchasing process ### Acceptance Criteria 1. The user can see a list of all products in the cart, including: - Product name, image - Product unit price - Purchase quantity - Subtotal 2. The system automatically calculates the total order amount 3. The user can choose/edit the shipping address 4. The user can choose the payment method 5. The user can use coupons 6. The user can fill in order remarks 7. The order confirmation page is displayed before submitting the order 8. After successfully submitting the order, the user is redirected to the payment page ## 2. Direct Purchase from Product Details Page **As** a registered user **I want** to purchase products directly from the product details page **so that** I can quickly complete the purchase of a single product ### Acceptance Criteria 1. The user can select from the product details page: - Product specifications (such as color, size, etc.) - Purchase quantity 2. After clicking "Buy Now", the user is taken to the order confirmation page 3. The user can choose/edit the shipping address 4. The user can choose the payment method 5. The user can use coupons 6. The user can fill in order remarks 7. The order confirmation page is displayed before submitting the order 8. After successfully submitting the order, the user is redirected to the payment page ## 3. Order Confirmation **As** a registered user **I want** to confirm the order information before submitting **so that** I can ensure the order information is accurate ### Acceptance Criteria 1. The order confirmation page displays: - Recipient information (name, phone, address) - Product list (name, specifications, quantity, unit price) - Total order amount - Shipping cost - Discount information - Amount payable 2. The user can edit the shipping address 3. The user can choose/change the coupon 4. The user can choose the payment method 5. The user can fill in/edit order remarks 6. The estimated delivery time is displayed 7. A "Submit Order" button is provided ## 4. Order Payment **As** a registered user **I want** to complete the order payment **so that** I can confirm the purchase and wait for the product to be shipped ### Acceptance Criteria 1. The payment page displays: - Order number - Amount payable - Payment countdown (e.g., 15 minutes) 2. Multiple payment methods are provided: - Alipay - WeChat Pay - Quick bank card payment - Other payment methods 3. After successful payment: - Display the payment success page - View order details - Continue shopping 4. In case of payment failure: - Display the reason for failure - Provide retry options - Offer other payment method choices ## 5. Special Scenario Handling **As** the system **I want** to correctly handle special situations during the ordering process **so that** I can provide a good user experience and ensure system stability ### Acceptance Criteria 1. When stock is insufficient: - Prompt the user in a timely manner - Automatically adjust the purchasable quantity - Provide a notification function for arrival 2. When product prices change: - Update the price in real-time - Prompt the user about price changes 3. Coupon usage: - Check usage conditions - Display discount amounts - Prompt usage restrictions 4. When orders timeout: - Countdown reminder - Automatically cancel the order - Release inventory 5. In case of network anomalies: - Save order information - Provide a retry mechanism - Show friendly error prompts ## 6. Order Status Inquiry **As** a registered user **I want** to check the order status at any time **so that** I can understand the order progress ### Acceptance Criteria 1. The user can view: - Pending payment orders - Pending shipment orders - Pending receipt orders - Completed orders 2. The order details page displays: - Basic order information - Payment information - Logistics information - Receipt information 3. Provides order operation options: - Cancel order - Pay - Confirm receipt - Apply for refund 4. Provides order status change notifications ## 7. User Security Authentication **As** a system administrator **I want** to ensure the security of user identity authentication **so that** I can protect user accounts and transaction security ### Acceptance Criteria 1. Login security: - Supports multi-factor authentication (MFA) - Remote login requires SMS verification - Temporarily lock account after consecutive login failures - Supports fingerprint/face recognition and other biometric authentications 2. Password security: - Enforce complex password requirements (including uppercase and lowercase letters, numbers, and special characters) - Regularly remind users to update passwords - Password reset requires multi-step verification 3. Session management: - Automatically log out after inactivity timeout - Notify of remote login to the same account - Encrypt session tokens for storage 4. Login logs: - Record all login activities - Record login IP and device information - Notify of abnormal login behaviors ## 8. Payment Security **As** a user **I want** a secure and reliable payment environment **so that** I can confidently make online payments ### Acceptance Criteria 1. Payment information protection: - Bank card information is encrypted for storage - Payment passwords are set separately - Large payments require additional verification 2. Transaction security: - Payment environment detection - Device risk assessment - Real-time monitoring of suspicious transactions 3. Payment restrictions: - Single transaction limit - Daily transaction limit - Intercept suspicious transactions 4. Payment notifications: - Real-time payment result notifications - Abnormal transaction alerts - Account balance change notifications ## 9. Data Security and Privacy Protection **As** a user **I want** my personal information to be properly protected **so that** I can safely and securely use platform services ### Acceptance Criteria 1. Personal information protection: - Sensitive information is displayed in a desensitized manner - Data transmission is encrypted - Access permissions are controlled 2. Privacy settings: - Users can control the visibility of information - Set aliases for shipping addresses - Privacy policies are clearly communicated 3. Data storage: - Data is stored in a classified manner - Sensitive data is encrypted - Backup mechanisms for data 4. Third-party information sharing: - User authorization mechanisms - Control over the scope of information sharing - Sharing records are queryable ## 10. Risk Control **As** a system **I want** to establish a comprehensive risk control mechanism **so that** I can prevent and address various security risks ### Acceptance Criteria 1. Account risk control: - Monitor abnormal account behaviors - Risk level assessment - Account freezing mechanisms 2. Transaction risk control: - Analyze transaction behaviors - Anti-fraud rule engine - Intercept suspicious transactions 3. System security protection: - DDoS attack protection - SQL injection protection - XSS attack protection - URL tampering prevention mechanisms: - URL parameter encryption - Request signature verification - URL expiration control - Prevent URL replay attacks - Prevent CSRF attacks - Parameter legitimacy validation 4. Emergency response: - Security incident response processes - Emergency handling plans - Disaster recovery mechanisms
How Strong Is Windsurf in Writing Test Cases?
Figure 3. Windsurf User Story
# E-commerce System Test Cases ## 1. Cart Order Testing ### 1.1 Basic Functionality Testing of the Cart | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC1.1.1 | View Cart Product List | User is logged in, and there are products in the cart | 1. Enter the cart page | Display all product information (name, image, unit price, quantity, subtotal) || TC1.1.2 | Modify Product Quantity | There are products in the cart | 1. Click the increase quantity button<br/>2. Click the decrease quantity button<br/>3. Directly input quantity | 1. Quantity increases by 1<br/>2. Quantity decreases by 1<br/>3. Updates to input quantity<br/>4. Total price updates automatically || TC1.1.3 | Delete Product | There are products in the cart | 1. Click the delete button<br/>2. Confirm deletion | Product is removed from the cart, and total price updates |### 1.2 Order Confirmation Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC1.2.1 | Select Shipping Address | User has added multiple shipping addresses | 1. Click to select shipping address<br/>2. Choose one address | Successfully select the address and display it in the order || TC1.2.2 | Use Coupon | User has available coupons | 1. Click to select coupon<br/>2. Choose one coupon | Discount amount is correctly calculated and updates the total price || TC1.2.3 | Fill in Order Remarks | Entering the order confirmation page | 1. Input content in the remarks box | Remark information is successfully saved |## 2. Direct Purchase from Product Details Page Testing ### 2.1 Product Specification Selection Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC2.1.1 | Select Product Specifications | Product has multiple specification options | 1. Select color<br/>2. Select size | 1. Specification selection is successful<br/>2. Price and inventory update || TC2.1.2 | Inventory Check | Product inventory is limited | 1. Select quantity exceeding inventory | Display insufficient stock prompt |## 3. Payment Function Testing ### 3.1 Payment Method Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC3.1.1 | Alipay Payment | Order has been created | 1. Select Alipay payment<br/>2. Redirect to Alipay page<br/>3. Complete payment | Payment successful, order status updates || TC3.1.2 | WeChat Payment | Order has been created | 1. Select WeChat payment<br/>2. Display payment QR code<br/>3. Scan to pay | Payment successful, order status updates |### 3.2 Payment Exception Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC3.2.1 | Payment Timeout | Order has been created | 1. Select payment method<br/>2. Wait for payment timeout | Order is automatically canceled, inventory released || TC3.2.2 | Payment Failure | Order has been created | 1. Use an account with insufficient balance to pay | Display payment failure reason, can retry payment |## 4. Security Testing ### 4.1 User Authentication Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC4.1.1 | Multi-Factor Authentication | User has MFA enabled | 1. Input password to log in<br/>2. Input mobile verification code | Successfully log in after verification || TC4.1.2 | Remote Login Detection | User logs in on a new device | 1. Log in on a new device<br/>2. Input verification code | Send login notification, require verification |### 4.2 URL Security Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC4.2.1 | URL Parameter Tampering | User is logged in | 1. Modify the order ID in the URL<br/>2. Access the modified URL | Access denied, record security incident || TC4.2.2 | URL Replay Attack | Obtain valid URL | 1. Save URL<br/>2. Wait for URL to expire<br/>3. Access again | Prompt URL has expired, access denied |### 4.3 Payment Security Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC4.3.1 | Large Payment Verification | Order amount exceeds threshold | 1. Submit a large order<br/>2. Attempt payment | Requires additional identity verification || TC4.3.2 | Payment Environment Detection | User pays in a new environment | 1. Pay in an unsafe network environment | Prompt environment risk, suggest changing network |## 5. Performance Testing ### 5.1 Concurrent Order Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC5.1.1 | High-Concurrency Ordering | System is running normally | 1. Simulate 1000 users placing orders simultaneously | System responds normally, orders are created correctly || TC5.1.2 | Inventory Concurrent Processing | Product inventory is limited | 1. Multiple users purchase the same product simultaneously | Inventory is correctly deducted, no overselling occurs |### 5.2 Response Time Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC5.2.1 | Page Load Time | System is running normally | 1. Access the product list page<br/>2. Access the product details page | Page load time ≤ 3 seconds || TC5.2.2 | Order Processing Time | System is running normally | 1. Submit order<br/>2. Complete payment | Order processing time ≤ 1 second |## 6. Data Security Testing ### 6.1 Data Encryption Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC6.1.1 | Sensitive Information Transmission | User inputs bank card information | 1. Capture and analyze transmission data | Data is encrypted, and plain text cannot be viewed || TC6.1.2 | Data Storage Encryption | Access the database | 1. View sensitive information in the database | Sensitive information is stored in encrypted form |### 6.2 Access Control Testing | Test ID | Test Scenario | Preconditions | Test Steps | Expected Results ||---------|---------|---------|---------|---------|| TC6.2.1 | User Permission Verification | Regular user logs in | 1. Access admin functions | Access denied, prompt insufficient permissions || TC6.2.2 | Data Access Control | User accesses order information | 1. Access other users' orders | Access denied, can only view own orders |
How Strong Is Windsurf in Writing Test Cases?
Figure 4. Windsurf Test Case

Leave a Comment